-- Logs begin at Fri 2022-09-09 20:14:53 UTC, end at Fri 2022-09-09 21:31:55 UTC. -- Sep 09 20:45:36.259704 dv-cinder-831247-28-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 09 20:45:36.268566 dv-cinder-831247-28-iscsi devstack@keystone.service[81617]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 09 20:45:36.271120 dv-cinder-831247-28-iscsi devstack@keystone.service[81617]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Sep 9 20:45:36 2022] *** Sep 09 20:45:36.271256 dv-cinder-831247-28-iscsi devstack@keystone.service[81617]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 09 20:45:36.271369 dv-cinder-831247-28-iscsi devstack@keystone.service[81617]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 09 20:45:36.271473 dv-cinder-831247-28-iscsi devstack@keystone.service[81617]: nodename: dv-cinder-831247-28-iscsi Sep 09 20:45:36.271568 dv-cinder-831247-28-iscsi devstack@keystone.service[81617]: machine: x86_64 Sep 09 20:45:36.271676 dv-cinder-831247-28-iscsi devstack@keystone.service[81617]: clock source: unix Sep 09 20:45:36.271777 dv-cinder-831247-28-iscsi devstack@keystone.service[81617]: pcre jit disabled Sep 09 20:45:36.271879 dv-cinder-831247-28-iscsi devstack@keystone.service[81617]: detected number of CPU cores: 4 Sep 09 20:45:36.271981 dv-cinder-831247-28-iscsi devstack@keystone.service[81617]: current working directory: / Sep 09 20:45:36.272134 dv-cinder-831247-28-iscsi devstack@keystone.service[81617]: detected binary path: /usr/bin/uwsgi-core Sep 09 20:45:36.272248 dv-cinder-831247-28-iscsi devstack@keystone.service[81617]: your processes number limit is 47680 Sep 09 20:45:36.272346 dv-cinder-831247-28-iscsi devstack@keystone.service[81617]: your memory page size is 4096 bytes Sep 09 20:45:36.272470 dv-cinder-831247-28-iscsi devstack@keystone.service[81617]: detected max file descriptor number: 2048 Sep 09 20:45:36.272470 dv-cinder-831247-28-iscsi devstack@keystone.service[81617]: lock engine: pthread robust mutexes Sep 09 20:45:36.272642 dv-cinder-831247-28-iscsi devstack@keystone.service[81617]: thunder lock: enabled Sep 09 20:45:36.272808 dv-cinder-831247-28-iscsi devstack@keystone.service[81617]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 09 20:45:36.272908 dv-cinder-831247-28-iscsi devstack@keystone.service[81617]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 09 20:45:36.312449 dv-cinder-831247-28-iscsi devstack@keystone.service[81617]: Python main interpreter initialized at 0x5624773a59c0 Sep 09 20:45:36.312449 dv-cinder-831247-28-iscsi devstack@keystone.service[81617]: python threads support enabled Sep 09 20:45:36.312449 dv-cinder-831247-28-iscsi devstack@keystone.service[81617]: your server socket listen backlog is limited to 100 connections Sep 09 20:45:36.312449 dv-cinder-831247-28-iscsi devstack@keystone.service[81617]: your mercy for graceful operations on workers is 90 seconds Sep 09 20:45:36.312787 dv-cinder-831247-28-iscsi devstack@keystone.service[81617]: mapped 403077 bytes (393 KB) for 2 cores Sep 09 20:45:36.312863 dv-cinder-831247-28-iscsi devstack@keystone.service[81617]: *** Operational MODE: preforking *** Sep 09 20:45:36.313122 dv-cinder-831247-28-iscsi devstack@keystone.service[81617]: *** uWSGI is running in multiple interpreter mode *** Sep 09 20:45:36.313122 dv-cinder-831247-28-iscsi devstack@keystone.service[81617]: spawned uWSGI master process (pid: 81617) Sep 09 20:45:36.313313 dv-cinder-831247-28-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 09 20:45:36.316107 dv-cinder-831247-28-iscsi devstack@keystone.service[81617]: spawned uWSGI worker 1 (pid: 81629, cores: 1) Sep 09 20:45:36.316107 dv-cinder-831247-28-iscsi devstack@keystone.service[81617]: spawned uWSGI worker 2 (pid: 81630, cores: 1) Sep 09 20:45:36.316107 dv-cinder-831247-28-iscsi devstack@keystone.service[81617]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 09 20:45:40.663791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 20:45:40.663791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: from cryptography.utils import int_from_bytes Sep 09 20:45:40.664601 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 20:45:40.664601 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: from cryptography.utils import int_from_bytes Sep 09 20:45:40.665723 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 20:45:40.665723 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: from cryptography.utils import int_from_bytes Sep 09 20:45:40.666197 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 20:45:40.666197 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: from cryptography.utils import int_from_bytes Sep 09 20:45:40.714890 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 09 20:45:40.715086 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 09 20:45:40.715281 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] command line args: [] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 09 20:45:40.715552 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 09 20:45:40.715732 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ================================================================================ {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 09 20:45:40.715986 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] admin_token = **** {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.716256 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 09 20:45:40.716352 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] config_dir = [] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.716432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.716511 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 09 20:45:40.716613 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] config_source = [] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.716693 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] command line args: [] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 09 20:45:40.716794 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] debug = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.716881 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 09 20:45:40.716995 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.717267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ================================================================================ {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 09 20:45:40.717332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] admin_token = **** {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.717396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.717547 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.717670 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] config_dir = [] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.717785 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.717900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.718004 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] config_source = [] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.718107 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.718210 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] debug = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.718315 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.718443 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] list_limit = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.718443 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] log_config_append = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.718628 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.718731 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.718846 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.718953 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] log_dir = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.719055 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.719158 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] log_file = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.719265 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.719393 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.719516 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] list_limit = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.719640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.719751 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] log_config_append = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.719872 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.719949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.720090 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.720186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] log_dir = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.720260 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.720386 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] log_file = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.720469 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.720549 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.720673 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.720762 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.720841 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.720913 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.721030 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.721105 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.721230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.721318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.721397 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.721474 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.721596 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.721678 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.721807 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.721887 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.722044 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.722165 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.722283 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.722364 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.722522 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.722759 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.722840 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.48/identity {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.722964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] publish_errors = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.723051 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.723202 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.723284 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.723435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.723513 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.723614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.723861 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.723988 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.48/identity {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.724118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.724211 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] publish_errors = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.724352 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] standard_threads = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.724433 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.724616 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.724712 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.724812 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.724901 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.724974 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.725047 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.725150 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] use_journal = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.725289 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.725369 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] use_json = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.725491 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] use_stderr = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.725569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] standard_threads = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.725668 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] use_syslog = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.725747 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.725900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.725983 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.726147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.726231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.726328 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] use_journal = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.726406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.726506 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] use_json = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.726633 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.726713 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] use_stderr = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.726849 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] use_syslog = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.726925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.727082 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 20:45:40.727161 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.727337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.727417 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.727578 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.727660 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.727821 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.727899 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.728089 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.728171 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.728345 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.728433 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.728587 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.728670 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.728820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.729401 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.729598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.729676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.729838 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.729915 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.730087 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.730166 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.730338 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.730415 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.730574 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.730658 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.730824 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.730908 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.731066 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.731143 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.731311 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.731386 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.731550 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.731630 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.731790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.731868 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.732051 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.732164 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.732331 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.732410 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.732582 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.732659 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.732813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.732896 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.733047 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.733136 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.733279 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.733364 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.733517 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.733601 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.733770 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.733851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.734020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.734098 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.734253 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.734340 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.734483 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.734561 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.734721 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.734804 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.734970 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.735059 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.735211 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.735295 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.735463 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.735542 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.735699 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.735785 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.735924 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.736000 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.736197 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.736279 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.736434 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.736559 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.736660 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.736757 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.736889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.736995 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.737122 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.737237 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.737363 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.737446 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.737597 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.737679 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.737833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.737917 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.738059 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.738139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.738296 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.738378 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.738532 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.738662 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.738770 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.738851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.739007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.739085 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.739241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.739327 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.739475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.739556 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.739712 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.739791 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.739950 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.740035 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.740213 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.740296 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.740450 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.740527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.740680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.740755 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.740923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.741004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.741159 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.741236 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.741397 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.741475 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.741630 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.741709 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.741862 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.741940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.742115 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.742193 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.742346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.742435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.742585 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.742663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.742820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.742897 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.743082 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.743169 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.743321 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.743398 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.743556 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.743633 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.743801 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.743888 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.744039 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.744134 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.744319 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.744423 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.744645 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.745231 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.745368 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.745612 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.745698 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.745853 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.746093 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.746175 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.746333 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.746532 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.746616 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.746811 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.746975 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.747055 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.747280 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.747411 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.747506 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.747739 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.747827 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.748003 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.748257 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.748353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.748522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.748657 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.748785 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.748962 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.749155 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.749286 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.749411 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.749536 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.749662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.749789 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.749928 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.750060 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.750150 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.750223 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.750302 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.750375 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.750457 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.750532 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.750783 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.750920 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.751049 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.751179 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.751327 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.751455 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.751585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.751717 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.751848 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.751991 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.752131 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.752255 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.752388 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.752515 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.752658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.752790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.752935 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.753047 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.753124 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] auth.external = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.753124 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.753263 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.753416 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.753606 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.753687 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.753872 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] auth.password = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.754039 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.754120 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] auth.token = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.754334 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.754470 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.754573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.754831 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.754990 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.755236 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.755499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.755780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.756001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.756162 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.756302 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.756384 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.756562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.756642 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.756802 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.756891 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.757045 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] credential.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.757125 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.757343 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.757423 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] auth.external = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.757589 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.757669 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.757845 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.757931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.758096 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.758176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.758346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.758427 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] auth.password = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.758581 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] auth.token = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.758660 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.758830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.758909 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.759069 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.759149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.759324 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.759402 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.759560 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.759638 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.759818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.759906 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.760094 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.760178 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.760397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.760479 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.760645 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.760909 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.760995 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] credential.caching = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.761149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] federation.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.761383 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.761537 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.761680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.761853 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.761948 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.762149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.762233 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.762517 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.762680 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.762770 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.763025 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.763166 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.763310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.763444 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.763587 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.763712 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.763836 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.764012 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.764178 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.764312 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.764429 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.764507 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.764580 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.764713 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.764844 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.764970 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.765133 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.765265 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.765394 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.765523 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.765653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.765795 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.765938 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.766069 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.766202 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.766309 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] federation.caching = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.766419 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.766549 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.766698 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.766833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.766966 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.767091 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.767215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.767297 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.767381 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.767456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.767578 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.767711 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.767972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.768060 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.768250 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.768519 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.768600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.768758 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.768938 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.769102 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.769346 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.769690 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.769812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.769946 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.770026 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.770186 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.770269 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] identity.caching = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.770430 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.770510 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.770695 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.770775 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.770943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.771033 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.771192 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.771273 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.771426 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.771505 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.771676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.771763 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.771923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.772004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.772183 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.772265 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.772437 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.772520 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.772681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.772760 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.772911 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.772989 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.773148 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.773228 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.773376 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.773456 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.773616 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.773696 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.773854 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.773936 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.774096 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.774175 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.774333 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.774413 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.774698 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.774813 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.774958 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.775187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.775302 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.775448 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.775706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.775787 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.775986 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.776092 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.776251 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.776333 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.776501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.776589 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.776746 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.776825 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.776991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.777071 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.777237 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.777318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.777472 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.777552 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.777717 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.777795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.777969 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.778050 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.778210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.778289 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.778460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.778539 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.778711 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.778792 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.778946 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.779031 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.779191 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.779269 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.779424 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.779504 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.779662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.779743 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.779902 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.779983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.780164 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.780250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.780401 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.780482 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.780655 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.780763 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.780908 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.781023 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.781159 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.781283 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.781410 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.781526 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.781660 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.781765 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.781896 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.782023 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.782147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.782280 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.782388 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.782522 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.782628 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.782764 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.782866 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.783007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.783088 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.783245 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.783324 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.783494 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.783573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.783761 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.783843 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.784000 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.784102 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.784282 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.784362 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.784535 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.784616 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.784777 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.784857 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.785015 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.785093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.785253 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.785334 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.785495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.785575 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.785738 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.785818 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.785972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.786052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.786205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.786284 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.786438 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] resource.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.786569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.786686 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.786769 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.786936 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.787015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.787174 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.787253 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.787428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.787510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.787665 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.787745 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.787907 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.787985 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.788174 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.788257 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.788416 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.788494 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.788656 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.788734 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.788893 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] role.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.788974 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.789145 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.789224 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] role.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.789378 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.789457 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.789630 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.789716 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.789885 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.789967 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.790124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.790203 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.790374 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.790454 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.790614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.790693 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.790857 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.790937 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.791100 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.791181 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.791348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.791968 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.792159 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.792275 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.792438 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.792520 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.792682 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.792763 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.792932 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] resource.caching = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.793011 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.793181 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.793262 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.793433 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.793599 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.793679 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.793841 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.793922 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.794097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.794176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.794353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.794434 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.794607 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.794689 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.794853 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.794932 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.795089 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.795168 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.795348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.795436 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] role.caching = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.795605 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.795689 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] role.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.795850 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.795930 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.796103 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.796187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.796346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.796426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.796581 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.796660 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.796838 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.796920 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.797098 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.797178 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.797350 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.797430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.797594 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.797682 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.797833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.797912 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.798068 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.798147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.798309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] token.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.798391 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.798552 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.798632 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.798802 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.798881 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.799044 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.799125 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.799304 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.799459 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.799559 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.799708 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.799795 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.799968 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.800048 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.800253 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.800338 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.800517 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.800611 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.800777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.800857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.801018 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.801097 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.801259 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.801340 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.801516 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.801653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.801790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.801881 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.802021 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.802110 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.802257 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.802337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.802495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.802610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.802753 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.802834 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.connection = **** {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.803003 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.803084 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.803254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.803334 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.803495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.803573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.803735 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.803817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.803967 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.804046 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.804231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.804312 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] token.caching = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.804471 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.804554 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.804715 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.804816 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.804954 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.805043 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.805192 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.805280 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.805439 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.805527 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.805678 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.805766 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.805927 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.806014 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.806162 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.806248 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.806411 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.806499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.806644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.806731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.806884 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.806970 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.807681 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.807778 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.807862 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 09 20:45:40.807938 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.808245 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.808483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.808987 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.809261 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.809526 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] database.connection = **** {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.809774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.810026 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.810269 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.810520 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.810762 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.811003 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.811242 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.811485 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.811734 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.811977 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.812253 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.812497 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.812750 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.812991 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.813239 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.813476 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.813723 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.813964 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 20:45:40.814138 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 09 20:45:40.836988 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:40.837219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:40.837445 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:40.838667 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:40.838904 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:40.839133 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:40.845737 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:40.847383 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:40.874406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:40.874628 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:40.876154 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:40.876391 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:40.927493 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:40.927828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:40.975832 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:40.976087 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:40.976512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:40.976752 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.132246 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.132455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.132542 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.132618 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.132691 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.132764 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.132899 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.133027 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.133112 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.133189 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.133324 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.133411 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.156508 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.156741 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.156866 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.156953 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.157107 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.157209 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.157341 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.157426 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.157571 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.157655 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.157792 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.157877 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.158009 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.158231 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.169478 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.169728 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.182141 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.182449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.182535 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.182681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.182765 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.182904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.182987 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.183124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.183207 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.183338 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.183428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.183553 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.183898 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.184032 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 20:45:41.200630 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.200842 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 20:45:41.201108 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.201193 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.201324 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 20:45:41.201409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 20:45:41.201630 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.201727 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.201859 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 20:45:41.201942 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.202137 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.202219 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.202397 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.202479 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.202659 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.202921 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.212010 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.212320 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.212587 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.212671 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.212870 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.212954 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.213144 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.213975 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.220369 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.220976 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.223259 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.223551 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.223842 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.223924 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.224157 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.224243 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.224448 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.224521 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.224711 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.224796 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.224982 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.225065 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.225264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.225365 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.225538 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.225691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.225827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.225958 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.226098 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.226230 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.226361 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.226595 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.249236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.249854 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.250703 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.250995 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.251325 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.251619 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.255658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.255946 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.256256 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.256386 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.256592 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.256878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.262608 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.262901 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.263230 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.263524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.268499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.268799 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.269092 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.269226 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.269385 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.269516 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.269673 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.269812 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.269945 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.270086 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.270218 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.270348 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.270485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.270624 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.270821 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.271095 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.286396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.286684 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.287077 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.287371 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.290547 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.291268 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.294348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.294655 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.295101 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.295405 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.298845 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.299644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.302086 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.302812 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.305333 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.305638 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.305924 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.306147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.306287 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.306460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.306591 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.306751 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.306881 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.307066 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.307350 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.307642 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.322763 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.323059 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.323330 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.323596 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.323812 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.324132 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.324410 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.324678 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.330431 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.331531 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.331977 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.333141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.333544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.333825 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.334680 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.334973 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.336540 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.336830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.337123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.337412 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.337666 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.337795 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.337937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.338762 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.339090 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.339514 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.349390 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.349682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.349968 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.350251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.350630 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.350763 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.350920 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.351062 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.351193 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.351325 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.351521 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.351905 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.352224 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.352513 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.367021 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.368435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.369982 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.371397 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.372891 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.374361 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.374492 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.375808 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.375938 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.376140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.376538 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.377488 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.377776 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.378183 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.382875 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.384363 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.384615 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.386117 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.387291 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.387692 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.387983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.388288 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.389043 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.389446 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.389737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.390015 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.396088 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.396388 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.396672 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.397868 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.398163 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.398450 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.404089 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.404387 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.404670 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.404949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.405232 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.405504 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.405783 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.405865 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 20:45:41.406073 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.406159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.406345 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.406427 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.406623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.406713 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.406896 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.406986 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.407175 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.407258 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.407443 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.407526 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.407711 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.407793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.407985 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.408280 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.408568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.408855 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.409123 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.409395 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 20:45:41.442250 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5624773a59c0 pid: 81629 (default app) Sep 09 20:45:41.444274 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5624773a59c0 pid: 81630 (default app) Sep 09 20:45:41.445456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb145e-42cc-48d7-837d-1d1e97de5e4a None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:41.445751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb145e-42cc-48d7-837d-1d1e97de5e4a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:41.446026 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb145e-42cc-48d7-837d-1d1e97de5e4a None None] PATH_INFO: `/v3` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:41.450188 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1/1] 10.222.0.48 () {50 vars in 791 bytes} [Fri Sep 9 20:45:41 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:45:48.999239 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d6fdb-074e-426d-b9a1-e9bc05e406b3 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:48.999765 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d6fdb-074e-426d-b9a1-e9bc05e406b3 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:48.999849 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d6fdb-074e-426d-b9a1-e9bc05e406b3 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:49.005823 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1/2] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:45:48 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:45:49.011768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973bbde-87f6-44a2-b51b-fa342f1076e7 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:49.011768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973bbde-87f6-44a2-b51b-fa342f1076e7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:49.012258 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973bbde-87f6-44a2-b51b-fa342f1076e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:49.082178 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: INFO dbcounter [None req-c973bbde-87f6-44a2-b51b-fa342f1076e7 None None] Registered counter for database keystone Sep 09 20:45:49.100977 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG oslo_db.sqlalchemy.engines [None req-c973bbde-87f6-44a2-b51b-fa342f1076e7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81629) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 09 20:45:49.107723 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writer thread running {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 09 20:45:49.298549 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG passlib.handlers.bcrypt [None req-c973bbde-87f6-44a2-b51b-fa342f1076e7 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81629) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 09 20:45:49.298849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG passlib.handlers.bcrypt [None req-c973bbde-87f6-44a2-b51b-fa342f1076e7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81629) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 09 20:45:49.356651 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c973bbde-87f6-44a2-b51b-fa342f1076e7 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:45:49.358246 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c973bbde-87f6-44a2-b51b-fa342f1076e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:45:49.485213 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 2/3] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:45:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 475 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 09 20:45:49.935141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d6fdb-074e-426d-b9a1-e9bc05e406b3 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:45:50.004184 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: INFO dbcounter [None req-862d6fdb-074e-426d-b9a1-e9bc05e406b3 None None] Registered counter for database keystone Sep 09 20:45:50.026485 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG oslo_db.sqlalchemy.engines [None req-862d6fdb-074e-426d-b9a1-e9bc05e406b3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81630) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 09 20:45:50.032742 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writer thread running {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 09 20:45:50.154756 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d6fdb-074e-426d-b9a1-e9bc05e406b3 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:45:50.159017 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376aaf8-eb52-4fe9-b317-10e7376526d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:45:50.159676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376aaf8-eb52-4fe9-b317-10e7376526d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:50.159995 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376aaf8-eb52-4fe9-b317-10e7376526d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:50.160332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376aaf8-eb52-4fe9-b317-10e7376526d5 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:50.168604 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1376aaf8-eb52-4fe9-b317-10e7376526d5 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:45:50.220383 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1376aaf8-eb52-4fe9-b317-10e7376526d5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 09 20:45:50.220670 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1376aaf8-eb52-4fe9-b317-10e7376526d5 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:45:50.229928 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 2/4] 10.222.0.48 () {58 vars in 1324 bytes} [Fri Sep 9 20:45:49 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 20:45:50.235878 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973bbde-87f6-44a2-b51b-fa342f1076e7 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:45:50.253072 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973bbde-87f6-44a2-b51b-fa342f1076e7 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:45:50.257435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d153888-c95e-47ad-b69d-70f959745901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:45:50.258549 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d153888-c95e-47ad-b69d-70f959745901 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:50.258831 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d153888-c95e-47ad-b69d-70f959745901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:50.259105 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d153888-c95e-47ad-b69d-70f959745901 admin admin] PATH_INFO: `/v3/projects/d43f38801344407291af57c99bc81066` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:50.261260 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d153888-c95e-47ad-b69d-70f959745901 admin admin] RBAC: Authorizing `identity:get_project(project_id=d43f38801344407291af57c99bc81066)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:45:50.313617 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7d153888-c95e-47ad-b69d-70f959745901 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 09 20:45:50.313962 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d153888-c95e-47ad-b69d-70f959745901 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:45:50.318405 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 3/5] 10.222.0.48 () {56 vars in 1207 bytes} [Fri Sep 9 20:45:50 2022] GET /identity/v3/projects/d43f38801344407291af57c99bc81066 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:45:52.432173 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607021c5-bd45-47d2-9ac8-9b70a214f866 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:52.432675 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607021c5-bd45-47d2-9ac8-9b70a214f866 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:52.432757 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607021c5-bd45-47d2-9ac8-9b70a214f866 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:52.437059 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 3/6] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:45:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:45:52.442359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f8989-a656-42bb-a2fa-696d15eff4b8 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:52.442759 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f8989-a656-42bb-a2fa-696d15eff4b8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:52.442902 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f8989-a656-42bb-a2fa-696d15eff4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:52.462434 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1f6f8989-a656-42bb-a2fa-696d15eff4b8 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:45:52.464535 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1f6f8989-a656-42bb-a2fa-696d15eff4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:45:52.472473 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 4/7] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:45:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 09 20:45:52.736354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607021c5-bd45-47d2-9ac8-9b70a214f866 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:45:52.748510 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607021c5-bd45-47d2-9ac8-9b70a214f866 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:45:52.752695 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0b3e6-5aa8-461a-9baf-15832078b7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:45:52.753283 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0b3e6-5aa8-461a-9baf-15832078b7d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:52.753559 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0b3e6-5aa8-461a-9baf-15832078b7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:52.753834 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0b3e6-5aa8-461a-9baf-15832078b7d6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:52.761892 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b0b3e6-5aa8-461a-9baf-15832078b7d6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:45:52.763646 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b0b3e6-5aa8-461a-9baf-15832078b7d6 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:45:52.772211 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 4/8] 10.222.0.48 () {58 vars in 1324 bytes} [Fri Sep 9 20:45:52 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 20:45:52.777827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f8989-a656-42bb-a2fa-696d15eff4b8 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:45:52.786634 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f8989-a656-42bb-a2fa-696d15eff4b8 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:45:52.790925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1873b086-8a36-4839-8665-1123a8e300cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:45:52.791576 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873b086-8a36-4839-8665-1123a8e300cc admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:52.791854 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873b086-8a36-4839-8665-1123a8e300cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:52.792159 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873b086-8a36-4839-8665-1123a8e300cc admin admin] PATH_INFO: `/v3/users/2e3a7a2557844cc78377f788a7c8c3ae` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:52.795860 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1873b086-8a36-4839-8665-1123a8e300cc admin admin] RBAC: Authorizing `identity:get_user(user_id=2e3a7a2557844cc78377f788a7c8c3ae)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:45:52.799639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1873b086-8a36-4839-8665-1123a8e300cc admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:45:52.801529 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 5/9] 10.222.0.48 () {56 vars in 1198 bytes} [Fri Sep 9 20:45:52 2022] GET /identity/v3/users/2e3a7a2557844cc78377f788a7c8c3ae => generated 247 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:45:55.005771 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3c30d-53a4-41ec-afeb-94734dcf9fb9 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:55.006328 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3c30d-53a4-41ec-afeb-94734dcf9fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:55.006328 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3c30d-53a4-41ec-afeb-94734dcf9fb9 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:55.007661 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 5/10] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:45:55.013322 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384aa911-0a3d-4661-a45f-c78a0e19a7ff None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:55.013742 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384aa911-0a3d-4661-a45f-c78a0e19a7ff None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:55.013877 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384aa911-0a3d-4661-a45f-c78a0e19a7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:55.031595 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-384aa911-0a3d-4661-a45f-c78a0e19a7ff None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:45:55.032798 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-384aa911-0a3d-4661-a45f-c78a0e19a7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:45:55.093822 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86ddfd-6d89-4d7f-95b2-4281595a4066 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:55.094106 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86ddfd-6d89-4d7f-95b2-4281595a4066 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:55.094380 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86ddfd-6d89-4d7f-95b2-4281595a4066 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:55.095691 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 6/11] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:45:55.100664 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca1814-5978-409f-869b-57920294564d None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:55.100947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca1814-5978-409f-869b-57920294564d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:55.101221 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca1814-5978-409f-869b-57920294564d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:55.115822 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 6/12] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:45:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:45:55.143120 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG passlib.handlers.bcrypt [None req-d6ca1814-5978-409f-869b-57920294564d None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81630) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 09 20:45:55.143503 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG passlib.handlers.bcrypt [None req-d6ca1814-5978-409f-869b-57920294564d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81630) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 09 20:45:55.192954 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-d6ca1814-5978-409f-869b-57920294564d None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:45:55.194501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d6ca1814-5978-409f-869b-57920294564d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:45:55.292167 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 7/13] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:45:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:45:55.386381 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384aa911-0a3d-4661-a45f-c78a0e19a7ff None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:45:55.410267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:45:55.490102 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384aa911-0a3d-4661-a45f-c78a0e19a7ff None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:45:55.522088 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6717bc-8db4-4664-9ae5-45ba472d306d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:45:55.522721 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6717bc-8db4-4664-9ae5-45ba472d306d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:55.523004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6717bc-8db4-4664-9ae5-45ba472d306d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:55.523279 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6717bc-8db4-4664-9ae5-45ba472d306d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:55.535059 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6717bc-8db4-4664-9ae5-45ba472d306d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:45:55.544262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-1f6717bc-8db4-4664-9ae5-45ba472d306d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:45:55.544262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:45:55.544262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:45:55.544262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:45:55.544262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:45:55.544262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:45:55.544262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:45:55.544262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:45:55.544262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:45:55.544262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:45:55.544262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:45:55.544262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:45:55.544262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:45:55.544262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:45:55.544262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:45:55.544262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:45:55.545265 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:45:55.545265 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:45:55.545265 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:45:55.545265 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:45:55.545265 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:45:55.545265 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:45:55.545265 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:45:55.545265 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:45:55.545265 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:45:55.545265 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:45:55.545265 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:45:55.545265 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:45:55.545265 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:45:55.545265 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:45:55.545265 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:45:55.545265 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:45:55.546500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:45:55.546500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:45:55.546500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:45:55.546500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:45:55.546500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:45:55.546500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:45:55.546500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:45:55.546500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:45:55.546500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:45:55.546500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:45:55.546500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:45:55.546500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:45:55.546500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 7/14] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:45:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:45:55.551170 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca1814-5978-409f-869b-57920294564d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:45:55.556585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6717bc-8db4-4664-9ae5-45ba472d306d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:45:55.637144 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:45:55.650720 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca1814-5978-409f-869b-57920294564d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:45:55.653062 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:45:55.655419 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6717bc-8db4-4664-9ae5-45ba472d306d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:45:55.688736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b7703a-6b76-4488-b912-aa42fb4fc01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:45:55.689602 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7703a-6b76-4488-b912-aa42fb4fc01c None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:55.689883 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7703a-6b76-4488-b912-aa42fb4fc01c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:55.690154 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7703a-6b76-4488-b912-aa42fb4fc01c None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:55.690358 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3250301-99d3-455a-8206-7e65bcfcccb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:45:55.690968 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3250301-99d3-455a-8206-7e65bcfcccb8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:55.691051 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b7703a-6b76-4488-b912-aa42fb4fc01c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:45:55.691252 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3250301-99d3-455a-8206-7e65bcfcccb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:55.691336 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b7703a-6b76-4488-b912-aa42fb4fc01c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:45:55.691527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3250301-99d3-455a-8206-7e65bcfcccb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:55.692740 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b7703a-6b76-4488-b912-aa42fb4fc01c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:45:55.704279 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 8/15] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:45:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:45:55.709875 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b7703a-6b76-4488-b912-aa42fb4fc01c None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:45:55.757999 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3250301-99d3-455a-8206-7e65bcfcccb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:45:55.759470 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3250301-99d3-455a-8206-7e65bcfcccb8 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:45:55.809523 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b7703a-6b76-4488-b912-aa42fb4fc01c None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:45:55.838347 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:45:55.842575 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b36697-e9ac-42b1-a87e-7e920c5fd5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:45:55.848731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b36697-e9ac-42b1-a87e-7e920c5fd5de None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:55.848731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b36697-e9ac-42b1-a87e-7e920c5fd5de None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:55.848731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b36697-e9ac-42b1-a87e-7e920c5fd5de None admin] PATH_INFO: `/v3/users/2e3a7a2557844cc78377f788a7c8c3ae` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:55.848731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b36697-e9ac-42b1-a87e-7e920c5fd5de None admin] RBAC: Authorizing `identity:get_user(user_id=2e3a7a2557844cc78377f788a7c8c3ae)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:45:55.848731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b36697-e9ac-42b1-a87e-7e920c5fd5de None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:45:55.848731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 9/16] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:45:55 2022] GET /identity/v3/users/2e3a7a2557844cc78377f788a7c8c3ae => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:45:55.853943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b36697-e9ac-42b1-a87e-7e920c5fd5de None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:45:55.856850 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 8/17] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:45:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 20:45:55.862445 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3250301-99d3-455a-8206-7e65bcfcccb8 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:45:55.888638 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:45:55.953882 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b36697-e9ac-42b1-a87e-7e920c5fd5de None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:45:55.958788 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3250301-99d3-455a-8206-7e65bcfcccb8 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:45:55.987431 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb582c4d-3a85-4ba7-9cfa-64881d9cfe4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:45:55.988062 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb582c4d-3a85-4ba7-9cfa-64881d9cfe4f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:55.988371 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb582c4d-3a85-4ba7-9cfa-64881d9cfe4f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:55.988640 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb582c4d-3a85-4ba7-9cfa-64881d9cfe4f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:55.989759 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb582c4d-3a85-4ba7-9cfa-64881d9cfe4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:45:55.991202 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb582c4d-3a85-4ba7-9cfa-64881d9cfe4f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:45:55.992786 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 10/18] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:45:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:45:55.993648 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276f909-843b-4bc4-8916-f1b46aa70033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:45:55.994311 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276f909-843b-4bc4-8916-f1b46aa70033 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:55.994588 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276f909-843b-4bc4-8916-f1b46aa70033 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:55.994857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276f909-843b-4bc4-8916-f1b46aa70033 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:55.998760 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb582c4d-3a85-4ba7-9cfa-64881d9cfe4f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:45:56.005160 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8276f909-843b-4bc4-8916-f1b46aa70033 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:45:56.006623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8276f909-843b-4bc4-8916-f1b46aa70033 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:45:56.022928 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-8276f909-843b-4bc4-8916-f1b46aa70033 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:45:56.022928 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:45:56.022928 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:45:56.022928 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:45:56.022928 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:45:56.022928 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:45:56.022928 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:45:56.022928 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:45:56.022928 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:45:56.022928 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:45:56.022928 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:45:56.022928 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:45:56.022928 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:45:56.022928 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:45:56.022928 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:45:56.022928 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:45:56.024341 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:45:56.024341 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:45:56.024341 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:45:56.024341 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:45:56.024341 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:45:56.024341 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:45:56.024341 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:45:56.024341 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:45:56.024341 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:45:56.024341 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:45:56.024341 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:45:56.024341 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:45:56.024341 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:45:56.024341 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:45:56.024341 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:45:56.024341 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:45:56.024341 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:45:56.025881 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:45:56.025881 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:45:56.025881 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:45:56.025881 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 9/19] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:45:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:45:56.032303 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276f909-843b-4bc4-8916-f1b46aa70033 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:45:56.066137 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:45:56.101898 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb582c4d-3a85-4ba7-9cfa-64881d9cfe4f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:45:56.103459 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:45:56.131935 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276f909-843b-4bc4-8916-f1b46aa70033 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:45:56.135930 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d41a1-b1e9-453d-93ce-f50048d6aae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:45:56.136571 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d41a1-b1e9-453d-93ce-f50048d6aae4 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:56.136850 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d41a1-b1e9-453d-93ce-f50048d6aae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:56.137121 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d41a1-b1e9-453d-93ce-f50048d6aae4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:56.137845 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d41a1-b1e9-453d-93ce-f50048d6aae4 None admin] RBAC: Adding query filter params (role.id=fcbf519fbc84458f913671a5a1d0914c, scope.domain.id=default, user.id=2e3a7a2557844cc78377f788a7c8c3ae) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:45:56.138153 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d41a1-b1e9-453d-93ce-f50048d6aae4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:45:56.139611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d41a1-b1e9-453d-93ce-f50048d6aae4 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:45:56.147556 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 11/20] 10.222.0.48 () {56 vars in 1322 bytes} [Fri Sep 9 20:45:55 2022] GET /identity/v3/role_assignments?user.id=2e3a7a2557844cc78377f788a7c8c3ae&scope.domain.id=default&role.id=fcbf519fbc84458f913671a5a1d0914c => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:45:56.171889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b898b-c0e3-4766-b7fc-1ac9123ec2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:45:56.172556 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b898b-c0e3-4766-b7fc-1ac9123ec2c9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:56.172837 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b898b-c0e3-4766-b7fc-1ac9123ec2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:56.173111 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b898b-c0e3-4766-b7fc-1ac9123ec2c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:56.173785 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781b898b-c0e3-4766-b7fc-1ac9123ec2c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:45:56.174107 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781b898b-c0e3-4766-b7fc-1ac9123ec2c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:45:56.175546 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781b898b-c0e3-4766-b7fc-1ac9123ec2c9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:45:56.194866 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 10/21] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:45:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:45:58.660033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10eb9e-9dbd-4c13-bbb3-f3a881137e8e None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:58.660619 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10eb9e-9dbd-4c13-bbb3-f3a881137e8e None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:58.660619 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10eb9e-9dbd-4c13-bbb3-f3a881137e8e None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:58.662190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 12/22] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:45:58.667324 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070aa24-d282-4c1b-8f98-ebcad54787e5 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:58.668162 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070aa24-d282-4c1b-8f98-ebcad54787e5 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:58.668650 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070aa24-d282-4c1b-8f98-ebcad54787e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:58.697058 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c070aa24-d282-4c1b-8f98-ebcad54787e5 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:45:58.699067 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c070aa24-d282-4c1b-8f98-ebcad54787e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:45:58.782264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 11/23] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:45:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:45:59.084037 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb26958-1f58-498a-b8b0-7231649d5603 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:59.084348 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb26958-1f58-498a-b8b0-7231649d5603 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:59.084863 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb26958-1f58-498a-b8b0-7231649d5603 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:59.088684 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 13/24] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:45:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:45:59.093796 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df46da6-b4b4-4ebd-887b-61d2e3b41632 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:59.094088 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df46da6-b4b4-4ebd-887b-61d2e3b41632 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:59.094366 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df46da6-b4b4-4ebd-887b-61d2e3b41632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:59.120049 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1df46da6-b4b4-4ebd-887b-61d2e3b41632 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:45:59.121273 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1df46da6-b4b4-4ebd-887b-61d2e3b41632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:45:59.127388 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876992e2-5fff-4e7b-8c38-534c14166e4b None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:59.127959 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876992e2-5fff-4e7b-8c38-534c14166e4b None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:59.128458 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876992e2-5fff-4e7b-8c38-534c14166e4b None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:59.130024 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 14/25] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:45:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:45:59.135542 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4f676-8781-4d5c-a108-ffa14300faa4 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:59.136132 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4f676-8781-4d5c-a108-ffa14300faa4 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:59.137113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4f676-8781-4d5c-a108-ffa14300faa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:59.161426 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:45:59.167825 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-56d4f676-8781-4d5c-a108-ffa14300faa4 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:45:59.169094 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-56d4f676-8781-4d5c-a108-ffa14300faa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:45:59.274712 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 12/26] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:45:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:45:59.276336 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df46da6-b4b4-4ebd-887b-61d2e3b41632 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:45:59.281171 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 15/27] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:45:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:45:59.289659 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b959641-9cdc-4f41-826c-41506103f1fb None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:59.290161 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b959641-9cdc-4f41-826c-41506103f1fb None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:59.290657 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b959641-9cdc-4f41-826c-41506103f1fb None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:59.293053 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 16/28] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:45:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:45:59.303459 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038491e-7327-458f-84cb-4e8cee3f3690 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:59.303991 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038491e-7327-458f-84cb-4e8cee3f3690 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:59.304508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038491e-7327-458f-84cb-4e8cee3f3690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:59.337434 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-2038491e-7327-458f-84cb-4e8cee3f3690 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:45:59.339508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2038491e-7327-458f-84cb-4e8cee3f3690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:45:59.376409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df46da6-b4b4-4ebd-887b-61d2e3b41632 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:45:59.420752 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e95445-f6b8-4ca4-9d88-73eda9ede2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:45:59.421655 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e95445-f6b8-4ca4-9d88-73eda9ede2fb None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:59.422089 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e95445-f6b8-4ca4-9d88-73eda9ede2fb None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:59.422508 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e95445-f6b8-4ca4-9d88-73eda9ede2fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:59.449531 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e95445-f6b8-4ca4-9d88-73eda9ede2fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:45:59.451379 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e95445-f6b8-4ca4-9d88-73eda9ede2fb None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:45:59.475856 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 17/29] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:45:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:45:59.484947 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-e5e95445-f6b8-4ca4-9d88-73eda9ede2fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:45:59.484947 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:45:59.484947 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:45:59.484947 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:45:59.484947 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:45:59.484947 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:45:59.484947 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:45:59.484947 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:45:59.484947 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:45:59.484947 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:45:59.484947 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:45:59.484947 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:45:59.484947 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:45:59.484947 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:45:59.484947 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:45:59.484947 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:45:59.485983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:45:59.485983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:45:59.485983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:45:59.485983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:45:59.485983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:45:59.485983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:45:59.485983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:45:59.485983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:45:59.485983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:45:59.485983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:45:59.485983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:45:59.485983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:45:59.485983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:45:59.485983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:45:59.485983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:45:59.485983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:45:59.485983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:45:59.487092 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:45:59.487092 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:45:59.487092 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:45:59.488619 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 13/30] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:45:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:45:59.494455 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2038491e-7327-458f-84cb-4e8cee3f3690 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:45:59.532904 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:45:59.627923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2038491e-7327-458f-84cb-4e8cee3f3690 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:45:59.674584 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3845bc-708f-4e17-90a4-6d3ca290e827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:45:59.675894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3845bc-708f-4e17-90a4-6d3ca290e827 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:59.676337 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3845bc-708f-4e17-90a4-6d3ca290e827 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:59.676768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3845bc-708f-4e17-90a4-6d3ca290e827 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:59.677591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3845bc-708f-4e17-90a4-6d3ca290e827 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:45:59.678036 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3845bc-708f-4e17-90a4-6d3ca290e827 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:45:59.679731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3845bc-708f-4e17-90a4-6d3ca290e827 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:45:59.707902 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 18/31] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:45:59.713780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e95445-f6b8-4ca4-9d88-73eda9ede2fb None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:45:59.723608 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:45:59.825276 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e95445-f6b8-4ca4-9d88-73eda9ede2fb None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:45:59.855756 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3845bc-708f-4e17-90a4-6d3ca290e827 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:45:59.877384 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68addf3c-ef2a-422d-aab5-43db6e381479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:45:59.878002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68addf3c-ef2a-422d-aab5-43db6e381479 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:45:59.878280 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68addf3c-ef2a-422d-aab5-43db6e381479 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:45:59.878550 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68addf3c-ef2a-422d-aab5-43db6e381479 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:45:59.880678 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68addf3c-ef2a-422d-aab5-43db6e381479 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:45:59.882182 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68addf3c-ef2a-422d-aab5-43db6e381479 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:45:59.970498 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3845bc-708f-4e17-90a4-6d3ca290e827 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:45:59.978977 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:00.013415 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758bdfd-83bf-4ec1-be1d-a57cf9e1fe4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:00.014087 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758bdfd-83bf-4ec1-be1d-a57cf9e1fe4d None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:00.014365 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758bdfd-83bf-4ec1-be1d-a57cf9e1fe4d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:00.014636 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758bdfd-83bf-4ec1-be1d-a57cf9e1fe4d None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:00.015325 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5758bdfd-83bf-4ec1-be1d-a57cf9e1fe4d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:00.016786 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5758bdfd-83bf-4ec1-be1d-a57cf9e1fe4d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:00.154091 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 14/32] 10.222.0.48 () {60 vars in 1141 bytes} [Fri Sep 9 20:45:59 2022] POST /identity/v3/projects => generated 301 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:46:00.155410 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68addf3c-ef2a-422d-aab5-43db6e381479 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:00.226354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:00.237895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 19/33] 10.222.0.48 () {60 vars in 1131 bytes} [Fri Sep 9 20:45:59 2022] POST /identity/v3/roles => generated 219 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:46:00.244737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758bdfd-83bf-4ec1-be1d-a57cf9e1fe4d None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:00.307982 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68addf3c-ef2a-422d-aab5-43db6e381479 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:00.383980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e10e4-49ed-4a8d-af82-0a721b735bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:00.384804 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e10e4-49ed-4a8d-af82-0a721b735bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:00.385133 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e10e4-49ed-4a8d-af82-0a721b735bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:00.385417 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e10e4-49ed-4a8d-af82-0a721b735bb8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:00.402441 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54e10e4-49ed-4a8d-af82-0a721b735bb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:00.409584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-e54e10e4-49ed-4a8d-af82-0a721b735bb8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:46:00.409584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:00.409584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:00.409584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:00.409584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:00.409584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:00.409584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:00.409584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:00.409584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:00.409584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:00.409584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:00.409584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:00.409584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:00.409584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:00.409584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:00.409584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:00.410496 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:00.410496 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:00.410496 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:00.410496 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:00.410496 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:00.410496 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:00.410496 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:00.410496 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:00.410496 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:00.410496 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:00.410496 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:00.410496 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:00.410496 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:00.410496 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:00.410496 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:00.410496 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:00.411264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:00.411264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:00.411264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:00.411264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:00.411264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:00.411264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:00.411264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:00.411264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:00.411264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:00.411264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:00.411264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:46:00.411264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:46:00.411264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 15/34] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:46:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:00.412813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba232d1a-1607-4fa0-acc0-48a62b2bb26f None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:00.412813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba232d1a-1607-4fa0-acc0-48a62b2bb26f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:00.412813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba232d1a-1607-4fa0-acc0-48a62b2bb26f None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:00.414478 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 16/35] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:00.421462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba232d1a-1607-4fa0-acc0-48a62b2bb26f None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:00.436505 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758bdfd-83bf-4ec1-be1d-a57cf9e1fe4d None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:00.477939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f629e9a-6ef1-48df-bc69-3a4bd682c1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:00.479147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f629e9a-6ef1-48df-bc69-3a4bd682c1ca None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:00.479437 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f629e9a-6ef1-48df-bc69-3a4bd682c1ca None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:00.479715 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f629e9a-6ef1-48df-bc69-3a4bd682c1ca None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:00.480692 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f629e9a-6ef1-48df-bc69-3a4bd682c1ca None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:00.482685 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f629e9a-6ef1-48df-bc69-3a4bd682c1ca None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:00.551324 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba232d1a-1607-4fa0-acc0-48a62b2bb26f None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:00.568901 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 20/36] 10.222.0.48 () {60 vars in 1131 bytes} [Fri Sep 9 20:46:00 2022] POST /identity/v3/roles => generated 225 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:46:00.571033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19df36-3e95-43fa-868c-18427f5635ab None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:00.571033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19df36-3e95-43fa-868c-18427f5635ab None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:00.571390 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19df36-3e95-43fa-868c-18427f5635ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:00.576147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:00.591268 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:00.598764 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-af19df36-3e95-43fa-868c-18427f5635ab None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:00.600129 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28202e1-5d3a-45c7-a5e8-a45886bfc6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:00.600936 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28202e1-5d3a-45c7-a5e8-a45886bfc6c4 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:00.601104 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-af19df36-3e95-43fa-868c-18427f5635ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:00.601544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28202e1-5d3a-45c7-a5e8-a45886bfc6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:00.601980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28202e1-5d3a-45c7-a5e8-a45886bfc6c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:00.602837 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28202e1-5d3a-45c7-a5e8-a45886bfc6c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:00.603882 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28202e1-5d3a-45c7-a5e8-a45886bfc6c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:00.607127 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28202e1-5d3a-45c7-a5e8-a45886bfc6c4 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:00.624744 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 17/37] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:00.630481 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28202e1-5d3a-45c7-a5e8-a45886bfc6c4 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:00.687665 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 21/38] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:00.689651 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ca32a-a5d1-4afb-9e6e-e54da3a2cafa None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:00.689947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ca32a-a5d1-4afb-9e6e-e54da3a2cafa None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:00.690297 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ca32a-a5d1-4afb-9e6e-e54da3a2cafa None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:00.691802 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 22/39] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:00.697425 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b784f-9787-494c-8a95-a555aa51ef27 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:00.697727 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b784f-9787-494c-8a95-a555aa51ef27 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:00.697988 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b784f-9787-494c-8a95-a555aa51ef27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:00.723379 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-fb8b784f-9787-494c-8a95-a555aa51ef27 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:00.725092 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-fb8b784f-9787-494c-8a95-a555aa51ef27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:00.766499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28202e1-5d3a-45c7-a5e8-a45886bfc6c4 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:00.810821 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 23/40] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:46:00.910874 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d68b63-a6ac-4052-95d5-3a5b5ba915b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:00.911765 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d68b63-a6ac-4052-95d5-3a5b5ba915b7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:00.912240 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d68b63-a6ac-4052-95d5-3a5b5ba915b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:00.912651 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d68b63-a6ac-4052-95d5-3a5b5ba915b7 None admin] PATH_INFO: `/v3/users/2e3a7a2557844cc78377f788a7c8c3ae` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:00.914007 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d68b63-a6ac-4052-95d5-3a5b5ba915b7 None admin] RBAC: Authorizing `identity:get_user(user_id=2e3a7a2557844cc78377f788a7c8c3ae)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:00.915588 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d68b63-a6ac-4052-95d5-3a5b5ba915b7 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:00.922538 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 18/41] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:00 2022] GET /identity/v3/users/2e3a7a2557844cc78377f788a7c8c3ae => generated 247 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:46:00.923745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b784f-9787-494c-8a95-a555aa51ef27 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:00.981183 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d68b63-a6ac-4052-95d5-3a5b5ba915b7 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:00.993785 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:01.029281 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b784f-9787-494c-8a95-a555aa51ef27 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:01.041759 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:01.064077 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3052cd0-6d11-482a-9266-7dfe04f7d6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:01.064856 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3052cd0-6d11-482a-9266-7dfe04f7d6f2 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:01.065230 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3052cd0-6d11-482a-9266-7dfe04f7d6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:01.065569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3052cd0-6d11-482a-9266-7dfe04f7d6f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:01.066732 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3052cd0-6d11-482a-9266-7dfe04f7d6f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:01.068351 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3052cd0-6d11-482a-9266-7dfe04f7d6f2 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:01.069980 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 24/42] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:46:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:01.075950 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3052cd0-6d11-482a-9266-7dfe04f7d6f2 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:01.102724 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d68b63-a6ac-4052-95d5-3a5b5ba915b7 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:01.135826 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e3e41d-790b-46bd-8ea8-7831e0bf5bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:01.136470 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e3e41d-790b-46bd-8ea8-7831e0bf5bcc None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:01.136753 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e3e41d-790b-46bd-8ea8-7831e0bf5bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:01.137026 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e3e41d-790b-46bd-8ea8-7831e0bf5bcc None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:01.137696 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e3e41d-790b-46bd-8ea8-7831e0bf5bcc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:01.139079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e3e41d-790b-46bd-8ea8-7831e0bf5bcc None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:01.177475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3052cd0-6d11-482a-9266-7dfe04f7d6f2 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:01.194416 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 19/43] 10.222.0.48 () {60 vars in 1131 bytes} [Fri Sep 9 20:46:00 2022] POST /identity/v3/roles => generated 223 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:46:01.195227 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e3e41d-790b-46bd-8ea8-7831e0bf5bcc None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:01.213487 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f85ef-1ede-4543-bb22-544c10056edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:01.214189 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f85ef-1ede-4543-bb22-544c10056edd None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:01.214618 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f85ef-1ede-4543-bb22-544c10056edd None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:01.214897 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f85ef-1ede-4543-bb22-544c10056edd None admin] PATH_INFO: `/v3/domains/default/users/2e3a7a2557844cc78377f788a7c8c3ae/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:01.217056 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405f85ef-1ede-4543-bb22-544c10056edd None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=2e3a7a2557844cc78377f788a7c8c3ae, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:01.218696 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405f85ef-1ede-4543-bb22-544c10056edd None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:01.221611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:01.308201 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 25/44] 10.222.0.48 () {58 vars in 1361 bytes} [Fri Sep 9 20:46:01 2022] PUT /identity/v3/domains/default/users/2e3a7a2557844cc78377f788a7c8c3ae/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:46:01.328475 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e3e41d-790b-46bd-8ea8-7831e0bf5bcc None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:01.340961 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:01.368608 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c8174-1dcd-44de-b8e0-bb9d16edef57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:01.369807 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c8174-1dcd-44de-b8e0-bb9d16edef57 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:01.370088 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c8174-1dcd-44de-b8e0-bb9d16edef57 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:01.370357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c8174-1dcd-44de-b8e0-bb9d16edef57 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:01.371701 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603c8174-1dcd-44de-b8e0-bb9d16edef57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:01.373330 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603c8174-1dcd-44de-b8e0-bb9d16edef57 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:01.374948 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 20/45] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:46:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:01.381720 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f85ef-1ede-4543-bb22-544c10056edd None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:01.499089 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f85ef-1ede-4543-bb22-544c10056edd None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:01.531898 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b6526-edd6-4b90-8a12-65cd60e76a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:01.532679 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b6526-edd6-4b90-8a12-65cd60e76a23 None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:01.532968 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b6526-edd6-4b90-8a12-65cd60e76a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:01.533245 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b6526-edd6-4b90-8a12-65cd60e76a23 None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:01.533911 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b6526-edd6-4b90-8a12-65cd60e76a23 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:01.535347 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b6526-edd6-4b90-8a12-65cd60e76a23 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:01.617003 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 26/46] 10.222.0.48 () {60 vars in 1141 bytes} [Fri Sep 9 20:46:01 2022] POST /identity/v3/projects => generated 312 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:46:03.486946 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b8a2c-f1bf-4c0b-9ef1-9c485f6baefd None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:03.487498 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b8a2c-f1bf-4c0b-9ef1-9c485f6baefd None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:03.487590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b8a2c-f1bf-4c0b-9ef1-9c485f6baefd None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:03.489308 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 21/47] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:03.495259 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eeaf71-5dbb-4bf5-a58e-62028fb4fbc4 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:03.495736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eeaf71-5dbb-4bf5-a58e-62028fb4fbc4 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:03.495956 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eeaf71-5dbb-4bf5-a58e-62028fb4fbc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:03.516315 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-d5eeaf71-5dbb-4bf5-a58e-62028fb4fbc4 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:03.517522 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d5eeaf71-5dbb-4bf5-a58e-62028fb4fbc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:03.558481 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:03.582390 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 27/48] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:46:03.778177 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22ae4c-d399-4056-8395-04805716a17b None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:03.778385 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22ae4c-d399-4056-8395-04805716a17b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:03.778674 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22ae4c-d399-4056-8395-04805716a17b None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:03.780011 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 22/49] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:03.785406 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7849eab-1d95-495b-ae08-8ba373841118 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:03.785705 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7849eab-1d95-495b-ae08-8ba373841118 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:03.785988 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7849eab-1d95-495b-ae08-8ba373841118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:03.803760 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-a7849eab-1d95-495b-ae08-8ba373841118 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:03.804997 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a7849eab-1d95-495b-ae08-8ba373841118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:03.852029 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22ae4c-d399-4056-8395-04805716a17b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:03.870478 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 28/50] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:03.955413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22ae4c-d399-4056-8395-04805716a17b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:03.988395 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56bd5ef-daa9-45c8-9916-dcbe829a8b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:03.989025 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bd5ef-daa9-45c8-9916-dcbe829a8b80 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:03.989309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bd5ef-daa9-45c8-9916-dcbe829a8b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:03.989583 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bd5ef-daa9-45c8-9916-dcbe829a8b80 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:04.000537 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56bd5ef-daa9-45c8-9916-dcbe829a8b80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:04.005377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-c56bd5ef-daa9-45c8-9916-dcbe829a8b80 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:46:04.005377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:04.005377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:04.005377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:04.005377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:04.005377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:04.005377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:04.005377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:04.005377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:04.005377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:04.005377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:04.005377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:04.005377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:04.005377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:04.005377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:04.005377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:04.006359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:04.006359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:04.006359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:04.006359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:04.006359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:04.006359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:04.006359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:04.006359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:04.006359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:04.006359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:04.006359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:04.006359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:04.006359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:04.006359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:04.006359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:04.006359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:04.007290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:04.007290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:04.007290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:04.007290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:04.007290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:04.007290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:04.007290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:04.007290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:04.007290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:04.007290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:04.007290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:46:04.007290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:46:04.007290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 23/51] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:46:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:46:04.013919 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7849eab-1d95-495b-ae08-8ba373841118 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:04.109556 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7849eab-1d95-495b-ae08-8ba373841118 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:04.116487 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:04.136910 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56bd5ef-daa9-45c8-9916-dcbe829a8b80 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:04.142378 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c672bf-b987-4773-8313-e63644f731a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:04.142987 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c672bf-b987-4773-8313-e63644f731a5 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:04.143270 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c672bf-b987-4773-8313-e63644f731a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:04.143544 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c672bf-b987-4773-8313-e63644f731a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:04.144261 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c672bf-b987-4773-8313-e63644f731a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:04.144569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c672bf-b987-4773-8313-e63644f731a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:04.145959 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c672bf-b987-4773-8313-e63644f731a5 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:04.154038 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 29/52] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:04.160035 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c672bf-b987-4773-8313-e63644f731a5 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:04.176693 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:04.240546 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56bd5ef-daa9-45c8-9916-dcbe829a8b80 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:04.258863 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c672bf-b987-4773-8313-e63644f731a5 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:04.274333 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e396c34-5ea2-4e1c-9675-a9927aa2f8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:04.274987 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e396c34-5ea2-4e1c-9675-a9927aa2f8ab None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:04.275293 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e396c34-5ea2-4e1c-9675-a9927aa2f8ab None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:04.275574 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e396c34-5ea2-4e1c-9675-a9927aa2f8ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:04.276716 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e396c34-5ea2-4e1c-9675-a9927aa2f8ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:04.278200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e396c34-5ea2-4e1c-9675-a9927aa2f8ab None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:04.279805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 24/53] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:46:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:46:04.285351 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e396c34-5ea2-4e1c-9675-a9927aa2f8ab None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:04.291368 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18cfb80-4c0d-4bed-9eb4-d20b5c64b472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:04.292002 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18cfb80-4c0d-4bed-9eb4-d20b5c64b472 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:04.292315 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18cfb80-4c0d-4bed-9eb4-d20b5c64b472 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:04.292589 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18cfb80-4c0d-4bed-9eb4-d20b5c64b472 None admin] PATH_INFO: `/v3/users/2e3a7a2557844cc78377f788a7c8c3ae` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:04.293727 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18cfb80-4c0d-4bed-9eb4-d20b5c64b472 None admin] RBAC: Authorizing `identity:get_user(user_id=2e3a7a2557844cc78377f788a7c8c3ae)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:04.295166 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18cfb80-4c0d-4bed-9eb4-d20b5c64b472 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:04.296763 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 30/54] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:04 2022] GET /identity/v3/users/2e3a7a2557844cc78377f788a7c8c3ae => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:46:04.302011 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18cfb80-4c0d-4bed-9eb4-d20b5c64b472 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:04.388338 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e396c34-5ea2-4e1c-9675-a9927aa2f8ab None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:04.390791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:04.426081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04cafc7-4668-45f6-96a5-d60a66a4b1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:04.426692 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cafc7-4668-45f6-96a5-d60a66a4b1b1 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:04.426994 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cafc7-4668-45f6-96a5-d60a66a4b1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:04.427267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cafc7-4668-45f6-96a5-d60a66a4b1b1 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:04.427920 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04cafc7-4668-45f6-96a5-d60a66a4b1b1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:04.429416 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04cafc7-4668-45f6-96a5-d60a66a4b1b1 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:04.439302 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:04.501567 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18cfb80-4c0d-4bed-9eb4-d20b5c64b472 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:04.533948 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a14ec-951a-4539-830b-8a2c60cc906b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:04.534576 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a14ec-951a-4539-830b-8a2c60cc906b None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:04.534859 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a14ec-951a-4539-830b-8a2c60cc906b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:04.535137 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a14ec-951a-4539-830b-8a2c60cc906b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:04.536269 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a14ec-951a-4539-830b-8a2c60cc906b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:04.537711 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a14ec-951a-4539-830b-8a2c60cc906b None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:04.539267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 31/55] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:46:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:46:04.544328 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 25/56] 10.222.0.48 () {60 vars in 1140 bytes} [Fri Sep 9 20:46:04 2022] POST /identity/v3/projects => generated 298 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:46:04.551248 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a14ec-951a-4539-830b-8a2c60cc906b None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:04.662149 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a14ec-951a-4539-830b-8a2c60cc906b None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:04.664661 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:04.698426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe88e8-83b1-45b4-b5b8-f5cdd67a4fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:04.699061 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe88e8-83b1-45b4-b5b8-f5cdd67a4fee None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:04.699343 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe88e8-83b1-45b4-b5b8-f5cdd67a4fee None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:04.699618 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe88e8-83b1-45b4-b5b8-f5cdd67a4fee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:04.700399 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe88e8-83b1-45b4-b5b8-f5cdd67a4fee None admin] RBAC: Adding query filter params (role.id=fcbf519fbc84458f913671a5a1d0914c, scope.domain.id=default, user.id=2e3a7a2557844cc78377f788a7c8c3ae) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:04.700712 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe88e8-83b1-45b4-b5b8-f5cdd67a4fee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:04.702169 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe88e8-83b1-45b4-b5b8-f5cdd67a4fee None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:04.708627 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 32/57] 10.222.0.48 () {56 vars in 1322 bytes} [Fri Sep 9 20:46:04 2022] GET /identity/v3/role_assignments?user.id=2e3a7a2557844cc78377f788a7c8c3ae&scope.domain.id=default&role.id=fcbf519fbc84458f913671a5a1d0914c => generated 530 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:06.726970 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02540744-0545-4690-91cd-456a9dd31fcf None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:06.727522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02540744-0545-4690-91cd-456a9dd31fcf None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:06.727522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02540744-0545-4690-91cd-456a9dd31fcf None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:06.728892 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 26/58] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:06.734062 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514226a-fac5-4bbf-af97-ba78ec0ef74e None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:06.734443 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514226a-fac5-4bbf-af97-ba78ec0ef74e None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:06.734626 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514226a-fac5-4bbf-af97-ba78ec0ef74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:06.752765 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-e514226a-fac5-4bbf-af97-ba78ec0ef74e None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:06.753956 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e514226a-fac5-4bbf-af97-ba78ec0ef74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:06.817867 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 33/59] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:07.089785 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02540744-0545-4690-91cd-456a9dd31fcf None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:07.188739 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02540744-0545-4690-91cd-456a9dd31fcf None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:07.220527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91668a25-4ffe-47a6-9baf-4087a7c85e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:07.221148 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91668a25-4ffe-47a6-9baf-4087a7c85e3c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:07.221428 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91668a25-4ffe-47a6-9baf-4087a7c85e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:07.221699 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91668a25-4ffe-47a6-9baf-4087a7c85e3c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:07.222787 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91668a25-4ffe-47a6-9baf-4087a7c85e3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:07.224260 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91668a25-4ffe-47a6-9baf-4087a7c85e3c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:07.225785 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 27/60] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:46:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:07.231679 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514226a-fac5-4bbf-af97-ba78ec0ef74e None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:07.322683 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:07.326291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514226a-fac5-4bbf-af97-ba78ec0ef74e None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:07.358289 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb00ba-b7b4-47ba-b90b-316233009005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:07.358925 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb00ba-b7b4-47ba-b90b-316233009005 None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:07.359206 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb00ba-b7b4-47ba-b90b-316233009005 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:07.359479 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb00ba-b7b4-47ba-b90b-316233009005 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:07.360147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb00ba-b7b4-47ba-b90b-316233009005 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:07.361596 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb00ba-b7b4-47ba-b90b-316233009005 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:07.456433 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 34/61] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Sep 9 20:46:07 2022] POST /identity/v3/users => generated 275 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:46:10.744871 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201299cd-fb69-49d5-8638-5679117cdf69 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:10.744871 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201299cd-fb69-49d5-8638-5679117cdf69 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:10.744871 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201299cd-fb69-49d5-8638-5679117cdf69 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:10.744871 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 28/62] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:10.757969 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791475c6-69ba-44bf-ad51-25c3dd71c77d None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:10.758941 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791475c6-69ba-44bf-ad51-25c3dd71c77d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:10.760049 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791475c6-69ba-44bf-ad51-25c3dd71c77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:10.789099 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-791475c6-69ba-44bf-ad51-25c3dd71c77d None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:10.790647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-791475c6-69ba-44bf-ad51-25c3dd71c77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:10.850588 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be5e05-692b-452f-a4e6-330e03c8bde0 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:10.850898 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be5e05-692b-452f-a4e6-330e03c8bde0 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:10.851152 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be5e05-692b-452f-a4e6-330e03c8bde0 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:10.852961 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 29/63] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:10.859631 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb4117-38b3-4091-b23d-dc8048bbbc6b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:10.859870 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb4117-38b3-4091-b23d-dc8048bbbc6b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:10.860127 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb4117-38b3-4091-b23d-dc8048bbbc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:10.881729 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 35/64] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:10.891177 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-92cb4117-38b3-4091-b23d-dc8048bbbc6b None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:10.892992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-92cb4117-38b3-4091-b23d-dc8048bbbc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:10.926078 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:11.002761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 30/65] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:46:11.060378 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa8027-808d-4043-8d9a-3640aeb05f96 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:11.061051 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa8027-808d-4043-8d9a-3640aeb05f96 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:11.062669 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa8027-808d-4043-8d9a-3640aeb05f96 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:11.064284 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 36/66] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:11.069647 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb3d2d-14aa-4833-a1ce-133c5e160032 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:11.070173 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb3d2d-14aa-4833-a1ce-133c5e160032 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:11.070675 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb3d2d-14aa-4833-a1ce-133c5e160032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:11.090216 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1afb3d2d-14aa-4833-a1ce-133c5e160032 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:11.091736 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1afb3d2d-14aa-4833-a1ce-133c5e160032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:11.172229 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa8027-808d-4043-8d9a-3640aeb05f96 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:11.178066 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 31/67] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:11.287979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:11.294604 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dfdb9-a3c4-42c0-93b1-becbcf74c05a None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:11.295162 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dfdb9-a3c4-42c0-93b1-becbcf74c05a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:11.295783 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dfdb9-a3c4-42c0-93b1-becbcf74c05a None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:11.297354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 32/68] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:46:11.302985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74ae23-8da3-4c0c-afc2-6f3130354b08 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:11.302985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74ae23-8da3-4c0c-afc2-6f3130354b08 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:11.302985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74ae23-8da3-4c0c-afc2-6f3130354b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:11.305489 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa8027-808d-4043-8d9a-3640aeb05f96 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:11.338025 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-7e74ae23-8da3-4c0c-afc2-6f3130354b08 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:11.346931 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7e74ae23-8da3-4c0c-afc2-6f3130354b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:11.347035 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440e6cf-8c38-4bf6-b373-43003c5ba2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:11.347035 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440e6cf-8c38-4bf6-b373-43003c5ba2d5 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:11.347035 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440e6cf-8c38-4bf6-b373-43003c5ba2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:11.347035 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440e6cf-8c38-4bf6-b373-43003c5ba2d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:11.354497 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440e6cf-8c38-4bf6-b373-43003c5ba2d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:11.363475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-b440e6cf-8c38-4bf6-b373-43003c5ba2d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:11.363475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:11.363475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:11.363475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:11.363475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:11.363475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:11.363475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:11.363475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:11.363475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:11.363475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:11.363475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:11.363475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:11.363475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:11.363475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:11.363475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:11.363475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:11.364607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:11.364607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:11.364607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:11.364607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:11.364607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:11.364607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:11.364607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:11.364607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:11.364607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:11.364607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:11.364607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:11.364607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:11.364607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:11.364607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:11.364607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:11.364607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:11.365370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:11.365370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:11.365370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:11.365370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:11.365370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:11.365370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:11.365370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:11.365370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:11.365370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:11.365370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:11.365370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:11.365370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:46:11.365370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 37/69] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:11.371107 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440e6cf-8c38-4bf6-b373-43003c5ba2d5 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:11.473959 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440e6cf-8c38-4bf6-b373-43003c5ba2d5 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:11.490021 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 33/70] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:11.491191 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74ae23-8da3-4c0c-afc2-6f3130354b08 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:11.508866 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:11.511608 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ff138-875e-466e-87d3-4ff906447e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:11.512261 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ff138-875e-466e-87d3-4ff906447e06 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:11.512561 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ff138-875e-466e-87d3-4ff906447e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:11.512823 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ff138-875e-466e-87d3-4ff906447e06 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:11.513505 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ff138-875e-466e-87d3-4ff906447e06 None admin] RBAC: Adding query filter params (name=member) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:11.513849 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ff138-875e-466e-87d3-4ff906447e06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:11.515213 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ff138-875e-466e-87d3-4ff906447e06 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:11.529530 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 38/71] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Sep 9 20:46:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:11.530499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ff138-875e-466e-87d3-4ff906447e06 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:11.540753 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:11.617026 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74ae23-8da3-4c0c-afc2-6f3130354b08 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:11.652654 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b19a835-6d05-4e62-b1d9-8717a7f0fd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:11.653296 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b19a835-6d05-4e62-b1d9-8717a7f0fd16 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:11.653580 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b19a835-6d05-4e62-b1d9-8717a7f0fd16 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:11.653858 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b19a835-6d05-4e62-b1d9-8717a7f0fd16 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:11.653956 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ff138-875e-466e-87d3-4ff906447e06 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:11.664516 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b19a835-6d05-4e62-b1d9-8717a7f0fd16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:11.669782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-1b19a835-6d05-4e62-b1d9-8717a7f0fd16 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:46:11.669782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:11.669782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:11.669782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:11.669782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:11.669782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:11.669782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:11.669782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:11.669782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:11.669782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:11.669782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:11.669782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:11.669782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:11.669782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:11.669782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:11.669782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:11.670658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:11.670658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:11.670658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:11.670658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:11.670658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:11.670658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:11.670658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:11.670658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:11.670658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:11.670658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:11.670658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:11.670658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:11.670658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:11.670658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:11.670658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:11.670658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:11.671437 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:11.671437 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:11.671437 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:11.671437 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:11.671437 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:11.671437 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:11.671437 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:11.671437 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:11.671437 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:11.671437 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:11.671437 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:46:11.671437 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:46:11.671437 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 34/72] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:46:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:11.677228 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b19a835-6d05-4e62-b1d9-8717a7f0fd16 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:11.697706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f99b8-0058-4da8-b1a2-091594422ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:11.698340 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f99b8-0058-4da8-b1a2-091594422ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:11.698621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f99b8-0058-4da8-b1a2-091594422ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:11.698893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f99b8-0058-4da8-b1a2-091594422ed9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:11.709010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f99b8-0058-4da8-b1a2-091594422ed9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:11.714525 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-b89f99b8-0058-4da8-b1a2-091594422ed9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:46:11.714525 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:11.714525 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:11.714525 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:11.714525 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:11.714525 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:11.714525 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:11.714525 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:11.714525 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:11.714525 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:11.714525 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:11.714525 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:11.714525 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:11.714525 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:11.714525 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:11.714525 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:11.715380 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:11.715380 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:11.715380 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:11.715380 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:11.715380 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:11.715380 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:11.715380 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:11.715380 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:11.715380 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:11.715380 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:11.715380 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:11.715380 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:11.715380 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:11.715380 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:11.715380 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:11.715380 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:11.716168 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:11.716168 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:11.716168 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:11.716168 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:11.716168 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:11.716168 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:11.716168 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:11.716168 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:11.716168 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:11.716168 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:11.716168 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:46:11.716168 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:46:11.716168 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 39/73] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Sep 9 20:46:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:11.717188 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f274f0-8f7a-4a8e-b165-0f66bedd8d2a None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:11.717469 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f274f0-8f7a-4a8e-b165-0f66bedd8d2a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:11.717743 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f274f0-8f7a-4a8e-b165-0f66bedd8d2a None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:11.719123 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 40/74] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:11.722135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f274f0-8f7a-4a8e-b165-0f66bedd8d2a None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:11.758329 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:11.783787 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b19a835-6d05-4e62-b1d9-8717a7f0fd16 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:11.791620 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:11.818582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b3acb-1f9a-4e22-9322-5961457ccdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:11.819218 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b3acb-1f9a-4e22-9322-5961457ccdfe None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:11.819500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b3acb-1f9a-4e22-9322-5961457ccdfe None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:11.819769 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b3acb-1f9a-4e22-9322-5961457ccdfe None admin] PATH_INFO: `/v3/users/2bca44a120f84b35b219ce7bed5a328a` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:11.827877 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f274f0-8f7a-4a8e-b165-0f66bedd8d2a None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:11.833116 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524b3acb-1f9a-4e22-9322-5961457ccdfe None admin] RBAC: Authorizing `identity:get_user(user_id=2bca44a120f84b35b219ce7bed5a328a)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:11.834567 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524b3acb-1f9a-4e22-9322-5961457ccdfe None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:11.836190 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 35/75] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:11 2022] GET /identity/v3/users/2bca44a120f84b35b219ce7bed5a328a => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:11.836944 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b3acb-1f9a-4e22-9322-5961457ccdfe None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:11.864677 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dd0189-b287-43a8-a03f-596fea713333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:11.865295 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd0189-b287-43a8-a03f-596fea713333 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:11.865675 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd0189-b287-43a8-a03f-596fea713333 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:11.865848 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd0189-b287-43a8-a03f-596fea713333 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:11.866513 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dd0189-b287-43a8-a03f-596fea713333 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:11.866822 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dd0189-b287-43a8-a03f-596fea713333 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:11.868240 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dd0189-b287-43a8-a03f-596fea713333 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:11.876552 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 41/76] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:11.877697 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09f8ec-01d7-4da6-b015-468648593250 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:11.877984 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09f8ec-01d7-4da6-b015-468648593250 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:11.878257 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09f8ec-01d7-4da6-b015-468648593250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:11.897913 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-3d09f8ec-01d7-4da6-b015-468648593250 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:11.899241 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3d09f8ec-01d7-4da6-b015-468648593250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:11.935155 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b3acb-1f9a-4e22-9322-5961457ccdfe None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:11.964178 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:11.964819 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 42/77] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:11.965655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09f8ec-01d7-4da6-b015-468648593250 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:11.977274 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c48e1-1c37-4ee5-a4e0-1b6a4da6a84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:11.984326 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c48e1-1c37-4ee5-a4e0-1b6a4da6a84e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:11.984326 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c48e1-1c37-4ee5-a4e0-1b6a4da6a84e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:11.984326 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c48e1-1c37-4ee5-a4e0-1b6a4da6a84e None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:11.984326 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c48e1-1c37-4ee5-a4e0-1b6a4da6a84e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:11.984326 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c48e1-1c37-4ee5-a4e0-1b6a4da6a84e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:11.984326 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c48e1-1c37-4ee5-a4e0-1b6a4da6a84e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:11.989232 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 36/78] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:46:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:11.989969 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c48e1-1c37-4ee5-a4e0-1b6a4da6a84e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:12.036627 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:12.081280 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09f8ec-01d7-4da6-b015-468648593250 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:12.096984 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c48e1-1c37-4ee5-a4e0-1b6a4da6a84e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:12.115482 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9e945-1aaa-42a2-a555-4f7fa8625163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:12.116138 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9e945-1aaa-42a2-a555-4f7fa8625163 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:12.116534 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9e945-1aaa-42a2-a555-4f7fa8625163 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:12.116809 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9e945-1aaa-42a2-a555-4f7fa8625163 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:12.127282 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a9e945-1aaa-42a2-a555-4f7fa8625163 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:12.132042 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-15a9e945-1aaa-42a2-a555-4f7fa8625163 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:12.132042 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:12.132042 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:12.132042 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:12.132042 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:12.132042 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:12.132042 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:12.132042 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:12.132042 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:12.132042 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:12.132042 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:12.132042 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:12.132042 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:12.132042 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:12.132042 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:12.132042 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:12.132906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:12.132906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:12.132906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:12.132906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:12.132906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:12.132906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:12.132906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:12.132906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:12.132906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:12.132906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:12.132906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:12.132906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:12.132906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:12.132906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:12.132906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:12.132906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:12.133667 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:12.133667 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:12.133667 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:12.133667 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:12.133667 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:12.133667 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:12.133667 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:12.133667 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:12.133667 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:12.133667 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:12.133667 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:12.133667 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:46:12.133667 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 43/79] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:12.134870 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9e945-1aaa-42a2-a555-4f7fa8625163 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:12.137212 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbd2437-0ac0-4464-a5e1-bd11fb827592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:12.138060 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd2437-0ac0-4464-a5e1-bd11fb827592 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:12.138809 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd2437-0ac0-4464-a5e1-bd11fb827592 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:12.139272 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd2437-0ac0-4464-a5e1-bd11fb827592 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:12.200031 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:12.207516 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbd2437-0ac0-4464-a5e1-bd11fb827592 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:12.209212 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbd2437-0ac0-4464-a5e1-bd11fb827592 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:12.239359 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9e945-1aaa-42a2-a555-4f7fa8625163 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:12.257179 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:12.273434 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4bc6f-57cb-4053-ab25-b4a6375381d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:12.274267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4bc6f-57cb-4053-ab25-b4a6375381d0 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:12.274546 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4bc6f-57cb-4053-ab25-b4a6375381d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:12.274859 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4bc6f-57cb-4053-ab25-b4a6375381d0 None admin] PATH_INFO: `/v3/users/2e3a7a2557844cc78377f788a7c8c3ae` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:12.276013 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4bc6f-57cb-4053-ab25-b4a6375381d0 None admin] RBAC: Authorizing `identity:get_user(user_id=2e3a7a2557844cc78377f788a7c8c3ae)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:12.277480 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4bc6f-57cb-4053-ab25-b4a6375381d0 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:12.279032 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 44/80] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:12 2022] GET /identity/v3/users/2e3a7a2557844cc78377f788a7c8c3ae => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:12.279786 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4bc6f-57cb-4053-ab25-b4a6375381d0 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:12.308863 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 37/81] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:46:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:46:12.309960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbd2437-0ac0-4464-a5e1-bd11fb827592 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:12.381480 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4bc6f-57cb-4053-ab25-b4a6375381d0 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:12.399316 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:12.406063 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbd2437-0ac0-4464-a5e1-bd11fb827592 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:12.413957 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a848b9bf-53b0-46fb-aec1-45728e7c16bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:12.414588 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848b9bf-53b0-46fb-aec1-45728e7c16bd None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:12.414882 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848b9bf-53b0-46fb-aec1-45728e7c16bd None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:12.415168 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848b9bf-53b0-46fb-aec1-45728e7c16bd None admin] PATH_INFO: `/v3/users/2bca44a120f84b35b219ce7bed5a328a` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:12.416834 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a848b9bf-53b0-46fb-aec1-45728e7c16bd None admin] RBAC: Authorizing `identity:get_user(user_id=2bca44a120f84b35b219ce7bed5a328a)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:12.418272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a848b9bf-53b0-46fb-aec1-45728e7c16bd None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:12.419851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 45/82] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:12 2022] GET /identity/v3/users/2bca44a120f84b35b219ce7bed5a328a => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:12.420663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a848b9bf-53b0-46fb-aec1-45728e7c16bd None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:12.440801 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2fd9ef-68f0-41da-86d7-dd9f2b617741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:12.441416 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fd9ef-68f0-41da-86d7-dd9f2b617741 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:12.441698 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fd9ef-68f0-41da-86d7-dd9f2b617741 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:12.442021 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fd9ef-68f0-41da-86d7-dd9f2b617741 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:12.442688 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2fd9ef-68f0-41da-86d7-dd9f2b617741 None admin] RBAC: Adding query filter params (name=member) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:12.443002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2fd9ef-68f0-41da-86d7-dd9f2b617741 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:12.444434 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2fd9ef-68f0-41da-86d7-dd9f2b617741 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:12.457665 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 38/83] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Sep 9 20:46:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:12.457665 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2fd9ef-68f0-41da-86d7-dd9f2b617741 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:12.480038 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:12.524943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a848b9bf-53b0-46fb-aec1-45728e7c16bd None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:12.550561 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2fd9ef-68f0-41da-86d7-dd9f2b617741 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:12.558873 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76056757-a1fe-4150-ad2f-893931b285e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:12.559758 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76056757-a1fe-4150-ad2f-893931b285e3 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:12.560033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76056757-a1fe-4150-ad2f-893931b285e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:12.560331 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76056757-a1fe-4150-ad2f-893931b285e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:12.561492 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76056757-a1fe-4150-ad2f-893931b285e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:12.562933 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76056757-a1fe-4150-ad2f-893931b285e3 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:12.564510 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 46/84] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:46:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:12.565267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76056757-a1fe-4150-ad2f-893931b285e3 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:12.588545 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c6e9df-83c6-4d5d-aa09-4233e7557a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:12.589148 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6e9df-83c6-4d5d-aa09-4233e7557a06 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:12.589428 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6e9df-83c6-4d5d-aa09-4233e7557a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:12.589699 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6e9df-83c6-4d5d-aa09-4233e7557a06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:12.616707 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:12.659812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c6e9df-83c6-4d5d-aa09-4233e7557a06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:12.661346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c6e9df-83c6-4d5d-aa09-4233e7557a06 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:12.662052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76056757-a1fe-4150-ad2f-893931b285e3 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:12.685145 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:12.696545 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be9dc2-e6e2-4443-9394-ed93ce6951d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:12.697181 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be9dc2-e6e2-4443-9394-ed93ce6951d1 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:12.697464 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be9dc2-e6e2-4443-9394-ed93ce6951d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:12.697739 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be9dc2-e6e2-4443-9394-ed93ce6951d1 None admin] PATH_INFO: `/v3/projects/1464b3dd76734fb593f3ed3582957e86` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:12.698886 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6be9dc2-e6e2-4443-9394-ed93ce6951d1 None admin] RBAC: Authorizing `identity:get_project(project_id=1464b3dd76734fb593f3ed3582957e86)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:12.700392 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6be9dc2-e6e2-4443-9394-ed93ce6951d1 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:12.702038 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 47/85] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:12 2022] GET /identity/v3/projects/1464b3dd76734fb593f3ed3582957e86 => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:12.702830 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be9dc2-e6e2-4443-9394-ed93ce6951d1 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:12.761269 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 39/86] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:46:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:46:12.762155 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c6e9df-83c6-4d5d-aa09-4233e7557a06 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:12.805274 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be9dc2-e6e2-4443-9394-ed93ce6951d1 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:12.817908 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:12.839143 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9c2d9-9f76-41db-9a0a-dd4761bd478a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:12.839747 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9c2d9-9f76-41db-9a0a-dd4761bd478a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:12.840030 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9c2d9-9f76-41db-9a0a-dd4761bd478a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:12.840344 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9c2d9-9f76-41db-9a0a-dd4761bd478a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:12.862006 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c6e9df-83c6-4d5d-aa09-4233e7557a06 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:12.897231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965e15e-43e2-4ca0-9605-9c868eec646f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:12.897869 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965e15e-43e2-4ca0-9605-9c868eec646f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:12.898155 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965e15e-43e2-4ca0-9605-9c868eec646f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:12.898434 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965e15e-43e2-4ca0-9605-9c868eec646f None admin] PATH_INFO: `/v3/users/2bca44a120f84b35b219ce7bed5a328a` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:12.899568 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965e15e-43e2-4ca0-9605-9c868eec646f None admin] RBAC: Authorizing `identity:get_user(user_id=2bca44a120f84b35b219ce7bed5a328a)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:12.901558 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965e15e-43e2-4ca0-9605-9c868eec646f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:12.902858 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:12.903337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 40/87] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:12 2022] GET /identity/v3/users/2bca44a120f84b35b219ce7bed5a328a => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:12.903886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965e15e-43e2-4ca0-9605-9c868eec646f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:12.917944 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c9c2d9-9f76-41db-9a0a-dd4761bd478a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:12.919446 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c9c2d9-9f76-41db-9a0a-dd4761bd478a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:13.006968 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965e15e-43e2-4ca0-9605-9c868eec646f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:13.027247 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 48/88] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:46:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:46:13.028141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9c2d9-9f76-41db-9a0a-dd4761bd478a None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:13.034729 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:13.048806 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65ef04-05a4-4913-8a7d-73df160f07c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:13.049508 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65ef04-05a4-4913-8a7d-73df160f07c9 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:13.050952 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65ef04-05a4-4913-8a7d-73df160f07c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:13.050952 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65ef04-05a4-4913-8a7d-73df160f07c9 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:13.050952 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e65ef04-05a4-4913-8a7d-73df160f07c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:13.052247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e65ef04-05a4-4913-8a7d-73df160f07c9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:13.124193 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:13.132891 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9c2d9-9f76-41db-9a0a-dd4761bd478a None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:13.159273 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 41/89] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Sep 9 20:46:12 2022] POST /identity/v3/users => generated 289 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:46:13.160201 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65ef04-05a4-4913-8a7d-73df160f07c9 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:13.166296 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f046a-8761-4306-a985-6ddd0ee60410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:13.166923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f046a-8761-4306-a985-6ddd0ee60410 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:13.167202 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f046a-8761-4306-a985-6ddd0ee60410 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:13.167476 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f046a-8761-4306-a985-6ddd0ee60410 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:13.169961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f046a-8761-4306-a985-6ddd0ee60410 None admin] RBAC: Adding query filter params (role.id=c59d891d042e48bab625f7b3666ef536, scope.project.id=1464b3dd76734fb593f3ed3582957e86, user.id=2bca44a120f84b35b219ce7bed5a328a) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:13.169961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f046a-8761-4306-a985-6ddd0ee60410 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:13.170137 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f046a-8761-4306-a985-6ddd0ee60410 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:13.176784 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 49/90] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:46:13 2022] GET /identity/v3/role_assignments?user.id=2bca44a120f84b35b219ce7bed5a328a&scope.project.id=1464b3dd76734fb593f3ed3582957e86&role.id=c59d891d042e48bab625f7b3666ef536 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:13.177734 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f046a-8761-4306-a985-6ddd0ee60410 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:13.326523 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65ef04-05a4-4913-8a7d-73df160f07c9 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:13.330411 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f046a-8761-4306-a985-6ddd0ee60410 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:13.378967 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13796699-6013-44ff-80b8-a35888f6172d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:13.380024 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13796699-6013-44ff-80b8-a35888f6172d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:13.380588 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13796699-6013-44ff-80b8-a35888f6172d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:13.381055 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13796699-6013-44ff-80b8-a35888f6172d None admin] PATH_INFO: `/v3/projects/1464b3dd76734fb593f3ed3582957e86` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:13.382462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13796699-6013-44ff-80b8-a35888f6172d None admin] RBAC: Authorizing `identity:get_project(project_id=1464b3dd76734fb593f3ed3582957e86)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:13.384846 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13796699-6013-44ff-80b8-a35888f6172d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:13.386785 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 42/91] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:13 2022] GET /identity/v3/projects/1464b3dd76734fb593f3ed3582957e86 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:13.387849 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784895b-fb3a-4788-90cd-20c91fc8f1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:13.387992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13796699-6013-44ff-80b8-a35888f6172d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:13.388550 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784895b-fb3a-4788-90cd-20c91fc8f1f8 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:13.388790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784895b-fb3a-4788-90cd-20c91fc8f1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:13.389066 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784895b-fb3a-4788-90cd-20c91fc8f1f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:13.418324 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:13.458652 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:13.472888 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784895b-fb3a-4788-90cd-20c91fc8f1f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:13.474390 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784895b-fb3a-4788-90cd-20c91fc8f1f8 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:13.519051 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13796699-6013-44ff-80b8-a35888f6172d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:13.565462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f75a8-91ee-438d-a9c3-085e99b6aac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:13.566117 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f75a8-91ee-438d-a9c3-085e99b6aac7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:13.566403 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f75a8-91ee-438d-a9c3-085e99b6aac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:13.566682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f75a8-91ee-438d-a9c3-085e99b6aac7 None admin] PATH_INFO: `/v3/projects/1464b3dd76734fb593f3ed3582957e86` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:13.567969 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f75a8-91ee-438d-a9c3-085e99b6aac7 None admin] RBAC: Authorizing `identity:get_project(project_id=1464b3dd76734fb593f3ed3582957e86)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:13.569505 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f75a8-91ee-438d-a9c3-085e99b6aac7 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:13.574278 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 43/92] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:13 2022] GET /identity/v3/projects/1464b3dd76734fb593f3ed3582957e86 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:13.574278 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f75a8-91ee-438d-a9c3-085e99b6aac7 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:13.608205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 50/93] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:46:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:46:13.609086 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784895b-fb3a-4788-90cd-20c91fc8f1f8 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:13.695694 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f75a8-91ee-438d-a9c3-085e99b6aac7 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:13.712585 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:13.722759 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:13.750130 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99dab44-9e98-4f6e-ac26-7c881396148b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:13.750758 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99dab44-9e98-4f6e-ac26-7c881396148b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:13.751039 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99dab44-9e98-4f6e-ac26-7c881396148b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:13.751311 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99dab44-9e98-4f6e-ac26-7c881396148b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:13.752055 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99dab44-9e98-4f6e-ac26-7c881396148b None admin] RBAC: Adding query filter params (role.id=fcbf519fbc84458f913671a5a1d0914c, scope.project.id=1464b3dd76734fb593f3ed3582957e86, user.id=2e3a7a2557844cc78377f788a7c8c3ae) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:13.756469 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99dab44-9e98-4f6e-ac26-7c881396148b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:13.757952 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99dab44-9e98-4f6e-ac26-7c881396148b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:13.766644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 44/94] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:46:13 2022] GET /identity/v3/role_assignments?user.id=2e3a7a2557844cc78377f788a7c8c3ae&scope.project.id=1464b3dd76734fb593f3ed3582957e86&role.id=fcbf519fbc84458f913671a5a1d0914c => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:13.767474 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99dab44-9e98-4f6e-ac26-7c881396148b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:13.782341 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784895b-fb3a-4788-90cd-20c91fc8f1f8 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:13.834539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3dc4e-76a5-4d77-b635-bd2911783eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:13.835165 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3dc4e-76a5-4d77-b635-bd2911783eb8 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:13.835430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3dc4e-76a5-4d77-b635-bd2911783eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:13.835717 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3dc4e-76a5-4d77-b635-bd2911783eb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:13.836469 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3dc4e-76a5-4d77-b635-bd2911783eb8 None admin] RBAC: Adding query filter params (role.id=13072ad4b81649459f59b8137db492ec, scope.project.id=1464b3dd76734fb593f3ed3582957e86, user.id=2bca44a120f84b35b219ce7bed5a328a) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:13.836782 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3dc4e-76a5-4d77-b635-bd2911783eb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:13.838289 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3dc4e-76a5-4d77-b635-bd2911783eb8 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:13.852644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 51/95] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:46:13 2022] GET /identity/v3/role_assignments?user.id=2bca44a120f84b35b219ce7bed5a328a&scope.project.id=1464b3dd76734fb593f3ed3582957e86&role.id=13072ad4b81649459f59b8137db492ec => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:13.991446 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99dab44-9e98-4f6e-ac26-7c881396148b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:14.034296 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c95b04-4185-427a-928c-d8de68eb18a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:14.034955 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c95b04-4185-427a-928c-d8de68eb18a4 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:14.035237 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c95b04-4185-427a-928c-d8de68eb18a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:14.035510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c95b04-4185-427a-928c-d8de68eb18a4 None admin] PATH_INFO: `/v3/projects/ae54b4a48e9f41c0a6d977e47550193f` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:14.036700 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c95b04-4185-427a-928c-d8de68eb18a4 None admin] RBAC: Authorizing `identity:get_project(project_id=ae54b4a48e9f41c0a6d977e47550193f)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:14.038180 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c95b04-4185-427a-928c-d8de68eb18a4 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:14.039870 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 45/96] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:13 2022] GET /identity/v3/projects/ae54b4a48e9f41c0a6d977e47550193f => generated 312 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:14.046047 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3dc4e-76a5-4d77-b635-bd2911783eb8 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:14.158510 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3dc4e-76a5-4d77-b635-bd2911783eb8 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:14.179447 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:14.207586 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4fad9-c962-4564-843b-77e0e8e7c896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:14.208260 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4fad9-c962-4564-843b-77e0e8e7c896 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:14.208539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4fad9-c962-4564-843b-77e0e8e7c896 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:14.208814 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4fad9-c962-4564-843b-77e0e8e7c896 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:14.209560 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f4fad9-c962-4564-843b-77e0e8e7c896 None admin] RBAC: Adding query filter params (role.id=c59d891d042e48bab625f7b3666ef536, scope.project.id=ae54b4a48e9f41c0a6d977e47550193f, user.id=2bca44a120f84b35b219ce7bed5a328a) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:14.209868 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f4fad9-c962-4564-843b-77e0e8e7c896 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:14.211312 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f4fad9-c962-4564-843b-77e0e8e7c896 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:14.217882 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 52/97] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:46:14 2022] GET /identity/v3/role_assignments?user.id=2bca44a120f84b35b219ce7bed5a328a&scope.project.id=ae54b4a48e9f41c0a6d977e47550193f&role.id=c59d891d042e48bab625f7b3666ef536 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:15.910980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfcf4ce-dadc-4259-b156-1b307a9d6020 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:15.911495 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfcf4ce-dadc-4259-b156-1b307a9d6020 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:15.911567 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfcf4ce-dadc-4259-b156-1b307a9d6020 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:15.913120 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 46/98] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:15.919217 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ea61f-babc-497d-8481-251a11c22e4f None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:15.920063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ea61f-babc-497d-8481-251a11c22e4f None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:15.920566 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ea61f-babc-497d-8481-251a11c22e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:15.949114 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-8b9ea61f-babc-497d-8481-251a11c22e4f None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:15.950671 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8b9ea61f-babc-497d-8481-251a11c22e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:16.054403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 53/99] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:16.500585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfcf4ce-dadc-4259-b156-1b307a9d6020 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:16.578286 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:16.640129 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfcf4ce-dadc-4259-b156-1b307a9d6020 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:16.658663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051409b8-a443-4a11-a7d0-bc2fcfbec9ed None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:16.659292 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051409b8-a443-4a11-a7d0-bc2fcfbec9ed None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:16.659759 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051409b8-a443-4a11-a7d0-bc2fcfbec9ed None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:16.661520 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 54/100] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:16.662723 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaed2c6-ccfd-4acd-bc46-39dac68fd09c None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:16.663214 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaed2c6-ccfd-4acd-bc46-39dac68fd09c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:16.663705 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaed2c6-ccfd-4acd-bc46-39dac68fd09c None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:16.665411 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 55/101] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:16.666632 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c9ace-a578-4636-ac2a-49e82a42b023 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:16.667105 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c9ace-a578-4636-ac2a-49e82a42b023 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:16.667568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c9ace-a578-4636-ac2a-49e82a42b023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:16.687454 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-791c9ace-a578-4636-ac2a-49e82a42b023 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:16.689010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-791c9ace-a578-4636-ac2a-49e82a42b023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:16.774305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd10fbf0-4462-44e3-9705-a58ed2dbec2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:16.774935 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10fbf0-4462-44e3-9705-a58ed2dbec2b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:16.775228 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10fbf0-4462-44e3-9705-a58ed2dbec2b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:16.775502 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10fbf0-4462-44e3-9705-a58ed2dbec2b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:16.789755 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd10fbf0-4462-44e3-9705-a58ed2dbec2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:16.798397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-fd10fbf0-4462-44e3-9705-a58ed2dbec2b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:16.798397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:16.798397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:16.798397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:16.798397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:16.798397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:16.798397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:16.798397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:16.798397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:16.798397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:16.798397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:16.798397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:16.798397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:16.798397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:16.798397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:16.798397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:16.799271 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:16.799271 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:16.799271 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:16.799271 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:16.799271 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:16.799271 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:16.799271 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:16.799271 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:16.799271 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:16.799271 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:16.799271 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:16.799271 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:16.799271 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:16.799271 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:16.799271 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:16.799271 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:16.800090 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:16.800090 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:16.800090 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:16.800090 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:16.800090 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:16.800090 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:16.800090 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:16.800090 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:16.800090 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:16.800090 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:16.800090 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:16.800090 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:46:16.800090 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 47/102] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:16.800754 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a947ba-d57a-472b-85eb-d6a913e1ff1b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:16.801646 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a947ba-d57a-472b-85eb-d6a913e1ff1b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:16.801646 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a947ba-d57a-472b-85eb-d6a913e1ff1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:16.817792 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 56/103] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:16.819431 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cfd6de-85d2-45de-9906-f30e0cc4b335 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:16.819962 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cfd6de-85d2-45de-9906-f30e0cc4b335 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:16.820485 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cfd6de-85d2-45de-9906-f30e0cc4b335 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:16.821985 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 57/104] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:16.822901 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cfd6de-85d2-45de-9906-f30e0cc4b335 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:16.828031 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c4a947ba-d57a-472b-85eb-d6a913e1ff1b None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:16.829316 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c4a947ba-d57a-472b-85eb-d6a913e1ff1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:16.872833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:16.935490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 48/105] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:16.937775 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf6e42-0cdd-46b7-bf97-36fbf52b0e8e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:16.940319 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf6e42-0cdd-46b7-bf97-36fbf52b0e8e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:16.940836 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf6e42-0cdd-46b7-bf97-36fbf52b0e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:16.991094 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-06bf6e42-0cdd-46b7-bf97-36fbf52b0e8e None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:16.994315 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-06bf6e42-0cdd-46b7-bf97-36fbf52b0e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:16.996558 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cfd6de-85d2-45de-9906-f30e0cc4b335 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:17.061713 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b9f46-0c08-426f-b82b-83491f13be14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:17.062669 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b9f46-0c08-426f-b82b-83491f13be14 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:17.063250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b9f46-0c08-426f-b82b-83491f13be14 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:17.063723 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b9f46-0c08-426f-b82b-83491f13be14 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:17.065674 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944b9f46-0c08-426f-b82b-83491f13be14 None admin] RBAC: Adding query filter params (name=member) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:17.067136 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944b9f46-0c08-426f-b82b-83491f13be14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:17.068768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944b9f46-0c08-426f-b82b-83491f13be14 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:17.069299 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:17.097249 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 58/106] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Sep 9 20:46:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:17.104436 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b9f46-0c08-426f-b82b-83491f13be14 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:17.155269 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 49/107] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:17.175302 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf6e42-0cdd-46b7-bf97-36fbf52b0e8e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:17.238807 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b9f46-0c08-426f-b82b-83491f13be14 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:17.241643 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:17.280266 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee32fd-a04a-461a-b56a-20685a0f58de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:17.280266 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee32fd-a04a-461a-b56a-20685a0f58de None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:17.280266 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee32fd-a04a-461a-b56a-20685a0f58de None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:17.280266 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee32fd-a04a-461a-b56a-20685a0f58de None admin] PATH_INFO: `/v3/users/2bca44a120f84b35b219ce7bed5a328a` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:17.280660 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dee32fd-a04a-461a-b56a-20685a0f58de None admin] RBAC: Authorizing `identity:get_user(user_id=2bca44a120f84b35b219ce7bed5a328a)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:17.280660 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dee32fd-a04a-461a-b56a-20685a0f58de None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:17.284006 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 59/108] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:17 2022] GET /identity/v3/users/2bca44a120f84b35b219ce7bed5a328a => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:17.287190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee32fd-a04a-461a-b56a-20685a0f58de None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:17.374397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf6e42-0cdd-46b7-bf97-36fbf52b0e8e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:17.412242 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aacb8ff-2dd4-4310-8567-eace2a67bc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:17.413459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacb8ff-2dd4-4310-8567-eace2a67bc49 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:17.413752 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacb8ff-2dd4-4310-8567-eace2a67bc49 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:17.414032 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacb8ff-2dd4-4310-8567-eace2a67bc49 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:17.426469 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee32fd-a04a-461a-b56a-20685a0f58de None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:17.427726 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aacb8ff-2dd4-4310-8567-eace2a67bc49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:17.433956 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-6aacb8ff-2dd4-4310-8567-eace2a67bc49 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:46:17.433956 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:17.433956 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:17.433956 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:17.433956 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:17.433956 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:17.433956 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:17.433956 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:17.433956 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:17.433956 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:17.433956 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:17.433956 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:17.433956 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:17.433956 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:17.433956 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:17.433956 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:17.435081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:17.435081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:17.435081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:17.435081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:17.435081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:17.435081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:17.435081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:17.435081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:17.435081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:17.435081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:17.435081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:17.435081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:17.435081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:17.435081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:17.435081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:17.435081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:17.435847 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:17.435847 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:17.435847 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:17.435847 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:17.435847 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:17.435847 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:17.435847 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:17.435847 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:17.435847 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:17.435847 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:17.435847 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:46:17.435847 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:46:17.435847 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 50/109] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Sep 9 20:46:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:17.437245 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aacb8ff-2dd4-4310-8567-eace2a67bc49 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:17.467454 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c43cf-00bc-40fe-a774-a80b0e1c76c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:17.468163 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c43cf-00bc-40fe-a774-a80b0e1c76c4 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:17.468500 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c43cf-00bc-40fe-a774-a80b0e1c76c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:17.468830 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c43cf-00bc-40fe-a774-a80b0e1c76c4 None admin] PATH_INFO: `/v3/projects/1464b3dd76734fb593f3ed3582957e86` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:17.469993 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c43cf-00bc-40fe-a774-a80b0e1c76c4 None admin] RBAC: Authorizing `identity:get_project(project_id=1464b3dd76734fb593f3ed3582957e86)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:17.471402 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c43cf-00bc-40fe-a774-a80b0e1c76c4 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:17.473925 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 60/110] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:17 2022] GET /identity/v3/projects/1464b3dd76734fb593f3ed3582957e86 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:17.476021 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:17.480005 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11db06-2b03-42d6-9f39-760103f746af None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:17.480321 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11db06-2b03-42d6-9f39-760103f746af None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:17.480686 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11db06-2b03-42d6-9f39-760103f746af None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:17.482730 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 61/111] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:17.486816 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11db06-2b03-42d6-9f39-760103f746af None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:17.536206 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:17.576868 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aacb8ff-2dd4-4310-8567-eace2a67bc49 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:17.599396 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11db06-2b03-42d6-9f39-760103f746af None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:17.612451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079db5df-63ff-479a-aa65-3b680f339d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:17.613142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079db5df-63ff-479a-aa65-3b680f339d90 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:17.613416 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079db5df-63ff-479a-aa65-3b680f339d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:17.613713 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079db5df-63ff-479a-aa65-3b680f339d90 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:17.614936 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079db5df-63ff-479a-aa65-3b680f339d90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:17.616503 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079db5df-63ff-479a-aa65-3b680f339d90 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:17.618315 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 51/112] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:46:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:17.618864 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079db5df-63ff-479a-aa65-3b680f339d90 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:17.639863 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4ae3e-e7e5-4696-aefa-d6a3965d6658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:17.640437 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4ae3e-e7e5-4696-aefa-d6a3965d6658 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:17.640723 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4ae3e-e7e5-4696-aefa-d6a3965d6658 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:17.641000 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4ae3e-e7e5-4696-aefa-d6a3965d6658 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:17.641690 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4ae3e-e7e5-4696-aefa-d6a3965d6658 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:17.642004 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4ae3e-e7e5-4696-aefa-d6a3965d6658 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:17.643481 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4ae3e-e7e5-4696-aefa-d6a3965d6658 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:17.654502 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 62/113] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:46:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:17.655603 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5688a-973d-4256-be8b-f0ef67694248 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:17.655897 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5688a-973d-4256-be8b-f0ef67694248 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:17.656214 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5688a-973d-4256-be8b-f0ef67694248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:17.677809 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-f9e5688a-973d-4256-be8b-f0ef67694248 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:17.679342 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f9e5688a-973d-4256-be8b-f0ef67694248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:17.729396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:17.735874 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079db5df-63ff-479a-aa65-3b680f339d90 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:17.752923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 63/114] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:17.752923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e5688a-973d-4256-be8b-f0ef67694248 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:17.775984 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df72943-adab-48dc-8a6b-9090bf28b2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:17.776682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df72943-adab-48dc-8a6b-9090bf28b2ab None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:17.776969 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df72943-adab-48dc-8a6b-9090bf28b2ab None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:17.777247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df72943-adab-48dc-8a6b-9090bf28b2ab None admin] PATH_INFO: `/v3/projects/1464b3dd76734fb593f3ed3582957e86/users/2bca44a120f84b35b219ce7bed5a328a/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:17.779173 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df72943-adab-48dc-8a6b-9090bf28b2ab None admin] RBAC: Authorizing `identity:create_grant(project_id=1464b3dd76734fb593f3ed3582957e86, user_id=2bca44a120f84b35b219ce7bed5a328a, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:17.780676 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df72943-adab-48dc-8a6b-9090bf28b2ab None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:17.791372 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:17.858110 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e5688a-973d-4256-be8b-f0ef67694248 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:17.866915 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 52/115] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Sep 9 20:46:17 2022] PUT /identity/v3/projects/1464b3dd76734fb593f3ed3582957e86/users/2bca44a120f84b35b219ce7bed5a328a/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:46:17.866915 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df72943-adab-48dc-8a6b-9090bf28b2ab None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:17.898245 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feded99-da4d-42f6-98b5-3abca9af41a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:17.898963 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feded99-da4d-42f6-98b5-3abca9af41a8 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:17.899257 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feded99-da4d-42f6-98b5-3abca9af41a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:17.899627 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feded99-da4d-42f6-98b5-3abca9af41a8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:17.911949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feded99-da4d-42f6-98b5-3abca9af41a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:17.917322 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-5feded99-da4d-42f6-98b5-3abca9af41a8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:46:17.917322 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:17.917322 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:17.917322 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:17.917322 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:17.917322 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:17.917322 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:17.917322 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:17.917322 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:17.917322 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:17.917322 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:17.917322 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:17.917322 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:17.917322 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:17.917322 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:17.917322 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:17.918334 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:17.918334 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:17.918334 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:17.918334 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:17.918334 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:17.918334 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:17.918334 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:17.918334 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:17.918334 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:17.918334 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:17.918334 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:17.918334 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:17.918334 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:17.918334 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:17.918334 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:17.918334 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:17.919121 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:17.919121 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:17.919121 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:17.919121 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:17.919121 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:17.919121 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:17.919121 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:17.919121 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:17.919121 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:17.919121 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:17.919121 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:46:17.919121 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:46:17.919121 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 64/116] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:17.919761 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feded99-da4d-42f6-98b5-3abca9af41a8 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:18.015893 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df72943-adab-48dc-8a6b-9090bf28b2ab None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:18.034298 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:18.044862 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feded99-da4d-42f6-98b5-3abca9af41a8 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:18.053311 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748d8aa-a4e3-4a6f-931f-d530be6bf429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:18.053933 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748d8aa-a4e3-4a6f-931f-d530be6bf429 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:18.054226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748d8aa-a4e3-4a6f-931f-d530be6bf429 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:18.054500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748d8aa-a4e3-4a6f-931f-d530be6bf429 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:18.055166 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5748d8aa-a4e3-4a6f-931f-d530be6bf429 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:18.057241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5748d8aa-a4e3-4a6f-931f-d530be6bf429 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:18.091302 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ead30-2af5-4a34-a3d4-53bcbbe72f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:18.092151 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ead30-2af5-4a34-a3d4-53bcbbe72f47 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:18.092416 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ead30-2af5-4a34-a3d4-53bcbbe72f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:18.092689 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ead30-2af5-4a34-a3d4-53bcbbe72f47 None admin] PATH_INFO: `/v3/users/2bca44a120f84b35b219ce7bed5a328a` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:18.094082 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ead30-2af5-4a34-a3d4-53bcbbe72f47 None admin] RBAC: Authorizing `identity:get_user(user_id=2bca44a120f84b35b219ce7bed5a328a)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:18.095633 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ead30-2af5-4a34-a3d4-53bcbbe72f47 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:18.097250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 65/117] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:17 2022] GET /identity/v3/users/2bca44a120f84b35b219ce7bed5a328a => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:18.098355 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd49c55-6183-4e60-a560-5b3626056fcd None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:18.098639 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd49c55-6183-4e60-a560-5b3626056fcd None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:18.098913 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd49c55-6183-4e60-a560-5b3626056fcd None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:18.100242 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 66/118] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:18.104305 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd49c55-6183-4e60-a560-5b3626056fcd None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:18.166822 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 53/119] 10.222.0.48 () {60 vars in 1141 bytes} [Fri Sep 9 20:46:17 2022] POST /identity/v3/projects => generated 302 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:46:18.167641 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748d8aa-a4e3-4a6f-931f-d530be6bf429 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:18.182958 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:18.230747 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd49c55-6183-4e60-a560-5b3626056fcd None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:18.282694 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd8b72-29e0-4c11-9827-86b35e2086c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:18.283321 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd8b72-29e0-4c11-9827-86b35e2086c1 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:18.283600 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd8b72-29e0-4c11-9827-86b35e2086c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:18.283876 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd8b72-29e0-4c11-9827-86b35e2086c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:18.285116 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd8b72-29e0-4c11-9827-86b35e2086c1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:18.285423 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd8b72-29e0-4c11-9827-86b35e2086c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:18.287204 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd8b72-29e0-4c11-9827-86b35e2086c1 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:18.300003 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748d8aa-a4e3-4a6f-931f-d530be6bf429 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:18.301581 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 67/120] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Sep 9 20:46:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:18.302788 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd8b72-29e0-4c11-9827-86b35e2086c1 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:18.340257 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:18.346017 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba4061-459b-4643-8bf1-073d62a41dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:18.346681 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba4061-459b-4643-8bf1-073d62a41dd8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:18.346958 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba4061-459b-4643-8bf1-073d62a41dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:18.347225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba4061-459b-4643-8bf1-073d62a41dd8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:18.364267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba4061-459b-4643-8bf1-073d62a41dd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:18.370562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-e0ba4061-459b-4643-8bf1-073d62a41dd8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:46:18.370562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:18.370562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:18.370562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:18.370562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:18.370562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:18.370562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:18.370562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:18.370562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:18.370562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:18.370562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:18.370562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:18.370562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:18.370562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:18.370562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:18.370562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:18.371620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:18.371620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:18.371620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:18.371620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:18.371620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:18.371620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:18.371620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:18.371620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:18.371620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:18.371620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:18.371620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:18.371620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:18.371620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:18.371620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:18.371620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:18.371620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:18.372682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:18.372682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:18.372682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:18.372682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:18.372682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:18.372682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:18.372682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:18.372682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:18.372682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:18.372682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:18.372682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:46:18.372682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:46:18.372682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 54/121] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:46:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:18.375226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31438185-97b6-4549-80a3-598dc45aadc7 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:18.375226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31438185-97b6-4549-80a3-598dc45aadc7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:18.375226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31438185-97b6-4549-80a3-598dc45aadc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:18.394766 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-31438185-97b6-4549-80a3-598dc45aadc7 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:18.396196 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-31438185-97b6-4549-80a3-598dc45aadc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:18.443284 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd8b72-29e0-4c11-9827-86b35e2086c1 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:18.470755 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:18.478382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 55/122] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:18.479227 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31438185-97b6-4549-80a3-598dc45aadc7 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:18.481298 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75427e45-0af0-46e2-8743-f1e8668eb7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:18.482168 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75427e45-0af0-46e2-8743-f1e8668eb7b2 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:18.482652 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75427e45-0af0-46e2-8743-f1e8668eb7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:18.483143 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75427e45-0af0-46e2-8743-f1e8668eb7b2 None admin] PATH_INFO: `/v3/projects/1464b3dd76734fb593f3ed3582957e86` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:18.484609 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75427e45-0af0-46e2-8743-f1e8668eb7b2 None admin] RBAC: Authorizing `identity:get_project(project_id=1464b3dd76734fb593f3ed3582957e86)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:18.486244 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75427e45-0af0-46e2-8743-f1e8668eb7b2 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:18.488250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 68/123] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:18 2022] GET /identity/v3/projects/1464b3dd76734fb593f3ed3582957e86 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:18.489174 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75427e45-0af0-46e2-8743-f1e8668eb7b2 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:18.652238 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31438185-97b6-4549-80a3-598dc45aadc7 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:18.673501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:18.687970 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741643ac-5f41-4ba2-99a8-647135f3a725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:18.688698 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741643ac-5f41-4ba2-99a8-647135f3a725 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:18.688982 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741643ac-5f41-4ba2-99a8-647135f3a725 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:18.689259 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741643ac-5f41-4ba2-99a8-647135f3a725 None admin] PATH_INFO: `/v3/users/57f1e4c0e3bb46ee9871ed96d96328fa` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:18.702288 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75427e45-0af0-46e2-8743-f1e8668eb7b2 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:18.703267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741643ac-5f41-4ba2-99a8-647135f3a725 None admin] RBAC: Authorizing `identity:get_user(user_id=57f1e4c0e3bb46ee9871ed96d96328fa)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:18.704796 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741643ac-5f41-4ba2-99a8-647135f3a725 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:18.706850 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 56/124] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:18 2022] GET /identity/v3/users/57f1e4c0e3bb46ee9871ed96d96328fa => generated 289 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:18.707716 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741643ac-5f41-4ba2-99a8-647135f3a725 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:18.768526 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:18.786894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f18f8-a7c8-4f55-9364-bac3fe2cced6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:18.787938 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f18f8-a7c8-4f55-9364-bac3fe2cced6 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:18.788457 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f18f8-a7c8-4f55-9364-bac3fe2cced6 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:18.788935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f18f8-a7c8-4f55-9364-bac3fe2cced6 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:18.789829 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f18f8-a7c8-4f55-9364-bac3fe2cced6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:18.790915 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f18f8-a7c8-4f55-9364-bac3fe2cced6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:18.792515 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f18f8-a7c8-4f55-9364-bac3fe2cced6 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:18.802889 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 69/125] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:18.804027 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f18f8-a7c8-4f55-9364-bac3fe2cced6 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:18.816928 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741643ac-5f41-4ba2-99a8-647135f3a725 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:18.860806 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7dd2a3-4abc-4364-8147-74ebc61c3cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:18.861503 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7dd2a3-4abc-4364-8147-74ebc61c3cb2 None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:18.861786 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7dd2a3-4abc-4364-8147-74ebc61c3cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:18.862063 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7dd2a3-4abc-4364-8147-74ebc61c3cb2 None admin] PATH_INFO: `/v3/projects/1464b3dd76734fb593f3ed3582957e86/users/2bca44a120f84b35b219ce7bed5a328a/roles/13072ad4b81649459f59b8137db492ec` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:18.864303 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7dd2a3-4abc-4364-8147-74ebc61c3cb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=1464b3dd76734fb593f3ed3582957e86, user_id=2bca44a120f84b35b219ce7bed5a328a, role_id=13072ad4b81649459f59b8137db492ec)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:18.865761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7dd2a3-4abc-4364-8147-74ebc61c3cb2 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:18.923620 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f18f8-a7c8-4f55-9364-bac3fe2cced6 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:18.960742 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec29a3e-29c0-4a95-a8bc-7f011654b1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:18.961699 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec29a3e-29c0-4a95-a8bc-7f011654b1b2 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:18.962730 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec29a3e-29c0-4a95-a8bc-7f011654b1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:18.963218 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec29a3e-29c0-4a95-a8bc-7f011654b1b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:18.984713 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 57/126] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Sep 9 20:46:18 2022] PUT /identity/v3/projects/1464b3dd76734fb593f3ed3582957e86/users/2bca44a120f84b35b219ce7bed5a328a/roles/13072ad4b81649459f59b8137db492ec => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:46:18.985721 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7dd2a3-4abc-4364-8147-74ebc61c3cb2 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:18.994233 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:19.053931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec29a3e-29c0-4a95-a8bc-7f011654b1b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:19.056572 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec29a3e-29c0-4a95-a8bc-7f011654b1b2 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:19.172646 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 70/127] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:46:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:46:19.173957 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec29a3e-29c0-4a95-a8bc-7f011654b1b2 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:19.189587 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7dd2a3-4abc-4364-8147-74ebc61c3cb2 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:19.220509 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:19.242955 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:19.249886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed12ff-4e57-4079-9da5-2500dcc7c146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:19.250537 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed12ff-4e57-4079-9da5-2500dcc7c146 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:19.250818 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed12ff-4e57-4079-9da5-2500dcc7c146 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:19.251102 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed12ff-4e57-4079-9da5-2500dcc7c146 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:19.263003 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed12ff-4e57-4079-9da5-2500dcc7c146 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:19.268965 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-e9ed12ff-4e57-4079-9da5-2500dcc7c146 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:19.268965 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:19.268965 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:19.268965 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:19.268965 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:19.268965 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:19.268965 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:19.268965 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:19.268965 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:19.268965 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:19.268965 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:19.268965 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:19.268965 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:19.268965 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:19.268965 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:19.268965 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:19.271991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:19.271991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:19.271991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:19.271991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:19.271991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:19.271991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:19.271991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:19.271991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:19.271991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:19.271991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:19.271991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:19.271991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:19.271991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:19.271991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:19.271991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:19.271991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:19.272822 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:19.272822 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:19.272822 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:19.272822 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:19.272822 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:19.272822 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:19.272822 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:19.272822 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:19.272822 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:19.272822 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:19.272822 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:19.272822 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:46:19.272822 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 58/128] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:19.272822 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed12ff-4e57-4079-9da5-2500dcc7c146 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:19.302597 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec29a3e-29c0-4a95-a8bc-7f011654b1b2 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:19.352483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3c2fc-bd3b-4394-997e-4f555f8c5ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:19.354102 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3c2fc-bd3b-4394-997e-4f555f8c5ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:19.354587 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3c2fc-bd3b-4394-997e-4f555f8c5ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:19.355062 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3c2fc-bd3b-4394-997e-4f555f8c5ae0 None admin] PATH_INFO: `/v3/users/2e3a7a2557844cc78377f788a7c8c3ae` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:19.357032 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e3c2fc-bd3b-4394-997e-4f555f8c5ae0 None admin] RBAC: Authorizing `identity:get_user(user_id=2e3a7a2557844cc78377f788a7c8c3ae)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:19.359031 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e3c2fc-bd3b-4394-997e-4f555f8c5ae0 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:19.365128 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 71/129] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:19 2022] GET /identity/v3/users/2e3a7a2557844cc78377f788a7c8c3ae => generated 247 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:19.366651 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3c2fc-bd3b-4394-997e-4f555f8c5ae0 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:19.439220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed12ff-4e57-4079-9da5-2500dcc7c146 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:19.484938 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08beeb-3f62-458e-830a-32852174e1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:19.485932 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08beeb-3f62-458e-830a-32852174e1e2 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:19.486235 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08beeb-3f62-458e-830a-32852174e1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:19.486514 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08beeb-3f62-458e-830a-32852174e1e2 None admin] PATH_INFO: `/v3/projects/1464b3dd76734fb593f3ed3582957e86` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:19.487734 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c08beeb-3f62-458e-830a-32852174e1e2 None admin] RBAC: Authorizing `identity:get_project(project_id=1464b3dd76734fb593f3ed3582957e86)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:19.489275 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c08beeb-3f62-458e-830a-32852174e1e2 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:19.490433 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3c2fc-bd3b-4394-997e-4f555f8c5ae0 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:19.491233 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 59/130] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:19 2022] GET /identity/v3/projects/1464b3dd76734fb593f3ed3582957e86 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:19.492025 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08beeb-3f62-458e-830a-32852174e1e2 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:19.517402 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:19.534323 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac782957-fe10-4fe7-92fe-d736091c1db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:19.535268 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac782957-fe10-4fe7-92fe-d736091c1db5 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:19.535756 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac782957-fe10-4fe7-92fe-d736091c1db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:19.536260 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac782957-fe10-4fe7-92fe-d736091c1db5 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:19.537134 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac782957-fe10-4fe7-92fe-d736091c1db5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:19.537639 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac782957-fe10-4fe7-92fe-d736091c1db5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:19.539233 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac782957-fe10-4fe7-92fe-d736091c1db5 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:19.549568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 72/131] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Sep 9 20:46:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:19.550692 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac782957-fe10-4fe7-92fe-d736091c1db5 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:19.590348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:19.668109 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac782957-fe10-4fe7-92fe-d736091c1db5 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:19.704835 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa04d68-e45f-4750-bcc6-f05795a58fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:19.705886 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa04d68-e45f-4750-bcc6-f05795a58fab None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:19.706372 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa04d68-e45f-4750-bcc6-f05795a58fab None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:19.706847 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa04d68-e45f-4750-bcc6-f05795a58fab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:19.707784 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa04d68-e45f-4750-bcc6-f05795a58fab None admin] RBAC: Adding query filter params (role.id=e01870d6deef4f1ebbc7470c342046a6, scope.project.id=1464b3dd76734fb593f3ed3582957e86, user.id=57f1e4c0e3bb46ee9871ed96d96328fa) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:19.708350 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa04d68-e45f-4750-bcc6-f05795a58fab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:19.710002 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa04d68-e45f-4750-bcc6-f05795a58fab None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:19.716793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 73/132] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:46:19 2022] GET /identity/v3/role_assignments?user.id=57f1e4c0e3bb46ee9871ed96d96328fa&scope.project.id=1464b3dd76734fb593f3ed3582957e86&role.id=e01870d6deef4f1ebbc7470c342046a6 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:19.717967 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa04d68-e45f-4750-bcc6-f05795a58fab None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:19.724110 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08beeb-3f62-458e-830a-32852174e1e2 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:19.805527 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:19.845114 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792fe51-8aa6-4b37-9e6f-6c3128d0f168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:19.845798 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792fe51-8aa6-4b37-9e6f-6c3128d0f168 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:19.846090 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792fe51-8aa6-4b37-9e6f-6c3128d0f168 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:19.846374 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792fe51-8aa6-4b37-9e6f-6c3128d0f168 None admin] PATH_INFO: `/v3/projects/1464b3dd76734fb593f3ed3582957e86` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:19.852046 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7792fe51-8aa6-4b37-9e6f-6c3128d0f168 None admin] RBAC: Authorizing `identity:get_project(project_id=1464b3dd76734fb593f3ed3582957e86)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:19.853737 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7792fe51-8aa6-4b37-9e6f-6c3128d0f168 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:19.855508 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 60/133] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:19 2022] GET /identity/v3/projects/1464b3dd76734fb593f3ed3582957e86 => generated 298 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:19.869191 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792fe51-8aa6-4b37-9e6f-6c3128d0f168 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:19.917299 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa04d68-e45f-4750-bcc6-f05795a58fab None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:20.002025 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41729f64-b01b-4c53-8cf0-3679c9089cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:20.002684 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41729f64-b01b-4c53-8cf0-3679c9089cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:20.002964 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41729f64-b01b-4c53-8cf0-3679c9089cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:20.003237 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41729f64-b01b-4c53-8cf0-3679c9089cf6 None admin] PATH_INFO: `/v3/users/2bca44a120f84b35b219ce7bed5a328a` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:20.013035 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41729f64-b01b-4c53-8cf0-3679c9089cf6 None admin] RBAC: Authorizing `identity:get_user(user_id=2bca44a120f84b35b219ce7bed5a328a)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:20.014687 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41729f64-b01b-4c53-8cf0-3679c9089cf6 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:20.016765 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 74/134] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:19 2022] GET /identity/v3/users/2bca44a120f84b35b219ce7bed5a328a => generated 275 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:20.022372 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41729f64-b01b-4c53-8cf0-3679c9089cf6 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:20.120939 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792fe51-8aa6-4b37-9e6f-6c3128d0f168 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:20.146802 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:20.195867 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41729f64-b01b-4c53-8cf0-3679c9089cf6 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:20.210741 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b989b4e-7e71-4eec-ae57-19d60c6a7e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:20.211431 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b989b4e-7e71-4eec-ae57-19d60c6a7e33 None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:20.211729 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b989b4e-7e71-4eec-ae57-19d60c6a7e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:20.212054 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b989b4e-7e71-4eec-ae57-19d60c6a7e33 None admin] PATH_INFO: `/v3/projects/1464b3dd76734fb593f3ed3582957e86/users/2e3a7a2557844cc78377f788a7c8c3ae/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:20.213195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:20.217584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b989b4e-7e71-4eec-ae57-19d60c6a7e33 None admin] RBAC: Authorizing `identity:create_grant(project_id=1464b3dd76734fb593f3ed3582957e86, user_id=2e3a7a2557844cc78377f788a7c8c3ae, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:20.219087 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b989b4e-7e71-4eec-ae57-19d60c6a7e33 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:20.236782 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79bbe51-f88b-4d94-bd27-89e09b3cda30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:20.237870 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79bbe51-f88b-4d94-bd27-89e09b3cda30 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:20.238429 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79bbe51-f88b-4d94-bd27-89e09b3cda30 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:20.238905 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79bbe51-f88b-4d94-bd27-89e09b3cda30 None admin] PATH_INFO: `/v3/projects/ae54b4a48e9f41c0a6d977e47550193f` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:20.240689 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79bbe51-f88b-4d94-bd27-89e09b3cda30 None admin] RBAC: Authorizing `identity:get_project(project_id=ae54b4a48e9f41c0a6d977e47550193f)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:20.242506 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79bbe51-f88b-4d94-bd27-89e09b3cda30 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:20.245277 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 75/135] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:20 2022] GET /identity/v3/projects/ae54b4a48e9f41c0a6d977e47550193f => generated 312 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:46:20.251356 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79bbe51-f88b-4d94-bd27-89e09b3cda30 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:20.337953 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 61/136] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Sep 9 20:46:19 2022] PUT /identity/v3/projects/1464b3dd76734fb593f3ed3582957e86/users/2e3a7a2557844cc78377f788a7c8c3ae/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:46:20.364863 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79bbe51-f88b-4d94-bd27-89e09b3cda30 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:20.403869 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a2569-ba9d-473a-87ad-1817bdbf3e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:20.404921 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a2569-ba9d-473a-87ad-1817bdbf3e46 None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:20.405381 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a2569-ba9d-473a-87ad-1817bdbf3e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:20.405835 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a2569-ba9d-473a-87ad-1817bdbf3e46 None admin] PATH_INFO: `/v3/projects/ae54b4a48e9f41c0a6d977e47550193f/users/2bca44a120f84b35b219ce7bed5a328a/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:20.408061 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a2569-ba9d-473a-87ad-1817bdbf3e46 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae54b4a48e9f41c0a6d977e47550193f, user_id=2bca44a120f84b35b219ce7bed5a328a, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:20.409858 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a2569-ba9d-473a-87ad-1817bdbf3e46 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:20.410711 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f977303-19bb-4a0a-b218-5ce498e41daa None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:20.411011 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f977303-19bb-4a0a-b218-5ce498e41daa None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:20.411292 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f977303-19bb-4a0a-b218-5ce498e41daa None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:20.415310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 62/137] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:20.424683 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2015701-6fd4-40a7-ba11-1f6fda2ce6d1 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:20.424970 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2015701-6fd4-40a7-ba11-1f6fda2ce6d1 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:20.427065 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2015701-6fd4-40a7-ba11-1f6fda2ce6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:20.457146 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c2015701-6fd4-40a7-ba11-1f6fda2ce6d1 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:20.458403 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c2015701-6fd4-40a7-ba11-1f6fda2ce6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:20.502144 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 76/138] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Sep 9 20:46:20 2022] PUT /identity/v3/projects/ae54b4a48e9f41c0a6d977e47550193f/users/2bca44a120f84b35b219ce7bed5a328a/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:46:20.554922 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 63/139] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:20.886297 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a4432-f639-499f-a0aa-9253547e75be None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:20.886901 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a4432-f639-499f-a0aa-9253547e75be None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:20.887403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a4432-f639-499f-a0aa-9253547e75be None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:20.889043 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 77/140] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:20.894557 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c23f9-8ee8-42f0-9c6f-ca15a5fe3e28 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:20.894853 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c23f9-8ee8-42f0-9c6f-ca15a5fe3e28 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:20.895139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c23f9-8ee8-42f0-9c6f-ca15a5fe3e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:20.921178 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a4432-f639-499f-a0aa-9253547e75be None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:20.923063 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c76c23f9-8ee8-42f0-9c6f-ca15a5fe3e28 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:20.924627 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c76c23f9-8ee8-42f0-9c6f-ca15a5fe3e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:20.981708 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:21.066355 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 64/141] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:21.088751 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a4432-f639-499f-a0aa-9253547e75be None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:21.132261 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eddde2-dde0-4f3c-9673-866de8ceee79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:21.133273 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eddde2-dde0-4f3c-9673-866de8ceee79 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:21.133808 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eddde2-dde0-4f3c-9673-866de8ceee79 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:21.134835 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eddde2-dde0-4f3c-9673-866de8ceee79 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:21.149301 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eddde2-dde0-4f3c-9673-866de8ceee79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:21.155076 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-d5eddde2-dde0-4f3c-9673-866de8ceee79 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:21.155076 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:21.155076 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:21.155076 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:21.155076 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:21.155076 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:21.155076 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:21.155076 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:21.155076 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:21.155076 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:21.155076 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:21.155076 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:21.155076 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:21.155076 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:21.155076 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:21.155076 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:21.156010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:21.156010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:21.156010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:21.156010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:21.156010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:21.156010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:21.156010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:21.156010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:21.156010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:21.156010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:21.156010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:21.156010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:21.156010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:21.156010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:21.156010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:21.156010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:21.156817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:21.156817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:21.156817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:21.156817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:21.156817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:21.156817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:21.156817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:21.156817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:21.156817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:21.156817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:21.156817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:21.156817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:46:21.159066 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 78/142] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:21.165089 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c23f9-8ee8-42f0-9c6f-ca15a5fe3e28 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:21.199928 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:21.277119 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c23f9-8ee8-42f0-9c6f-ca15a5fe3e28 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:21.323242 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e7333-bedd-4bd8-9744-7b9bc71db3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:21.324129 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e7333-bedd-4bd8-9744-7b9bc71db3e2 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:21.324570 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e7333-bedd-4bd8-9744-7b9bc71db3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:21.324989 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e7333-bedd-4bd8-9744-7b9bc71db3e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:21.325814 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e7333-bedd-4bd8-9744-7b9bc71db3e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:21.326265 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e7333-bedd-4bd8-9744-7b9bc71db3e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:21.327811 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e7333-bedd-4bd8-9744-7b9bc71db3e2 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:21.336465 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 65/143] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Sep 9 20:46:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:21.345164 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eddde2-dde0-4f3c-9673-866de8ceee79 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:21.476276 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eddde2-dde0-4f3c-9673-866de8ceee79 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:21.479919 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:21.520989 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cededfe3-7920-4987-bbe5-392b7d2391ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:21.521858 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cededfe3-7920-4987-bbe5-392b7d2391ae None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:21.522281 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cededfe3-7920-4987-bbe5-392b7d2391ae None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:21.522695 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cededfe3-7920-4987-bbe5-392b7d2391ae None admin] PATH_INFO: `/v3/users/2bca44a120f84b35b219ce7bed5a328a` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:21.528749 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cededfe3-7920-4987-bbe5-392b7d2391ae None admin] RBAC: Authorizing `identity:get_user(user_id=2bca44a120f84b35b219ce7bed5a328a)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:21.530375 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cededfe3-7920-4987-bbe5-392b7d2391ae None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:21.532289 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 79/144] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:21 2022] GET /identity/v3/users/2bca44a120f84b35b219ce7bed5a328a => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:21.537911 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e7333-bedd-4bd8-9744-7b9bc71db3e2 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:21.635676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cededfe3-7920-4987-bbe5-392b7d2391ae None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:21.683435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:21.686988 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e7333-bedd-4bd8-9744-7b9bc71db3e2 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:21.730195 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeaf80a-1ad0-4502-9725-b3deac2bbad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:21.731035 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeaf80a-1ad0-4502-9725-b3deac2bbad3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:21.731463 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeaf80a-1ad0-4502-9725-b3deac2bbad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:21.731884 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeaf80a-1ad0-4502-9725-b3deac2bbad3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:21.851521 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeaf80a-1ad0-4502-9725-b3deac2bbad3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:21.853794 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeaf80a-1ad0-4502-9725-b3deac2bbad3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:21.882007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cededfe3-7920-4987-bbe5-392b7d2391ae None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:21.955881 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd1009-d872-49af-bb95-c2862cf76089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:21.956559 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd1009-d872-49af-bb95-c2862cf76089 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:21.956842 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd1009-d872-49af-bb95-c2862cf76089 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:21.957120 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd1009-d872-49af-bb95-c2862cf76089 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:21.960811 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fd1009-d872-49af-bb95-c2862cf76089 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:21.962821 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fd1009-d872-49af-bb95-c2862cf76089 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:21.964801 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 80/145] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:46:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:21.970391 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd1009-d872-49af-bb95-c2862cf76089 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:22.001063 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:22.015124 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 66/146] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:46:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:46:22.020562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeaf80a-1ad0-4502-9725-b3deac2bbad3 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:22.052302 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:22.170952 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd1009-d872-49af-bb95-c2862cf76089 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:22.191882 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeaf80a-1ad0-4502-9725-b3deac2bbad3 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:22.249352 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaadb15-7b32-475b-a2e4-b88d89de5a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:22.249978 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaadb15-7b32-475b-a2e4-b88d89de5a8e None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:22.250258 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaadb15-7b32-475b-a2e4-b88d89de5a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:22.250526 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaadb15-7b32-475b-a2e4-b88d89de5a8e None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:22.251314 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaadb15-7b32-475b-a2e4-b88d89de5a8e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:22.251921 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dda401-9081-4382-a7d7-3f54e97df6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:22.252596 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dda401-9081-4382-a7d7-3f54e97df6af None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:22.252825 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaadb15-7b32-475b-a2e4-b88d89de5a8e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:22.252933 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dda401-9081-4382-a7d7-3f54e97df6af None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:22.253138 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dda401-9081-4382-a7d7-3f54e97df6af None admin] PATH_INFO: `/v3/projects/1464b3dd76734fb593f3ed3582957e86` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:22.254267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dda401-9081-4382-a7d7-3f54e97df6af None admin] RBAC: Authorizing `identity:get_project(project_id=1464b3dd76734fb593f3ed3582957e86)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:22.255821 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dda401-9081-4382-a7d7-3f54e97df6af None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:22.257467 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 67/147] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:22 2022] GET /identity/v3/projects/1464b3dd76734fb593f3ed3582957e86 => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:46:22.263647 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dda401-9081-4382-a7d7-3f54e97df6af None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:22.365785 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:22.387675 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 81/148] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Sep 9 20:46:21 2022] POST /identity/v3/users => generated 283 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:46:22.422888 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dda401-9081-4382-a7d7-3f54e97df6af None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:22.472683 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093670e6-91d2-42eb-8469-af6384f8433a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:22.473363 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093670e6-91d2-42eb-8469-af6384f8433a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:22.473643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093670e6-91d2-42eb-8469-af6384f8433a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:22.473914 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093670e6-91d2-42eb-8469-af6384f8433a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:22.474660 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093670e6-91d2-42eb-8469-af6384f8433a None admin] RBAC: Adding query filter params (role.id=c59d891d042e48bab625f7b3666ef536, scope.project.id=1464b3dd76734fb593f3ed3582957e86, user.id=2bca44a120f84b35b219ce7bed5a328a) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:22.475027 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093670e6-91d2-42eb-8469-af6384f8433a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:22.476524 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093670e6-91d2-42eb-8469-af6384f8433a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:22.488800 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 68/149] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:46:22 2022] GET /identity/v3/role_assignments?user.id=2bca44a120f84b35b219ce7bed5a328a&scope.project.id=1464b3dd76734fb593f3ed3582957e86&role.id=c59d891d042e48bab625f7b3666ef536 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:46:22.590264 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92989b6f-7330-44a8-bab7-6bc1002d11fe None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:22.590578 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92989b6f-7330-44a8-bab7-6bc1002d11fe None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:22.590744 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92989b6f-7330-44a8-bab7-6bc1002d11fe None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:22.592156 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 82/150] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:22.598084 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c45131-47a2-44b3-a95c-1e0060fc483c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:22.598374 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c45131-47a2-44b3-a95c-1e0060fc483c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:22.598653 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c45131-47a2-44b3-a95c-1e0060fc483c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:22.622923 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-90c45131-47a2-44b3-a95c-1e0060fc483c None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:22.624221 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-90c45131-47a2-44b3-a95c-1e0060fc483c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:22.716134 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 69/151] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:22.923548 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b184bbf-bc95-4e07-b8bf-8f18b456c4be None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:22.924267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b184bbf-bc95-4e07-b8bf-8f18b456c4be None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:22.924755 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b184bbf-bc95-4e07-b8bf-8f18b456c4be None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:22.926292 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 83/152] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:22.932319 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed6691-c1e2-49e2-9d26-b780fd0a2939 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:22.932608 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed6691-c1e2-49e2-9d26-b780fd0a2939 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:22.932945 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed6691-c1e2-49e2-9d26-b780fd0a2939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:22.954128 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-46ed6691-c1e2-49e2-9d26-b780fd0a2939 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:22.955317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-46ed6691-c1e2-49e2-9d26-b780fd0a2939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:23.017358 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:23.033938 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 70/153] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:23.077683 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b184bbf-bc95-4e07-b8bf-8f18b456c4be None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:23.183855 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:23.192812 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b184bbf-bc95-4e07-b8bf-8f18b456c4be None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:23.228816 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb404e-bab8-4e7c-bf17-b007bfc45462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:23.229467 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb404e-bab8-4e7c-bf17-b007bfc45462 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:23.229754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb404e-bab8-4e7c-bf17-b007bfc45462 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:23.230031 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb404e-bab8-4e7c-bf17-b007bfc45462 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:23.240503 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb404e-bab8-4e7c-bf17-b007bfc45462 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:23.245879 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-96cb404e-bab8-4e7c-bf17-b007bfc45462 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:46:23.245879 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:23.245879 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:23.245879 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:23.245879 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:23.245879 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:23.245879 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:23.245879 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:23.245879 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:23.245879 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:23.245879 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:23.245879 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:23.245879 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:23.245879 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:23.245879 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:23.245879 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:23.246878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:23.246878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:23.246878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:23.246878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:23.246878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:23.246878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:23.246878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:23.246878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:23.246878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:23.246878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:23.246878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:23.246878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:23.246878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:23.246878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:23.246878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:23.246878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:23.247818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:23.247818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:23.247818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:23.247818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:23.247818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:23.247818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:23.247818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:23.247818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:23.247818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:23.247818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:23.247818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:46:23.247818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:46:23.250465 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 84/154] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Sep 9 20:46:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:23.256691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ed6691-c1e2-49e2-9d26-b780fd0a2939 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:23.360363 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb404e-bab8-4e7c-bf17-b007bfc45462 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:23.379012 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ed6691-c1e2-49e2-9d26-b780fd0a2939 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:23.417130 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f4656d-e85d-441b-9b51-8afc33bb9a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:23.418146 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4656d-e85d-441b-9b51-8afc33bb9a55 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:23.418458 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4656d-e85d-441b-9b51-8afc33bb9a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:23.418737 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4656d-e85d-441b-9b51-8afc33bb9a55 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:23.419417 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f4656d-e85d-441b-9b51-8afc33bb9a55 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:23.419766 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f4656d-e85d-441b-9b51-8afc33bb9a55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:23.422156 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f4656d-e85d-441b-9b51-8afc33bb9a55 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:23.432851 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 71/155] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:46:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:23.434567 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784a210-d4cb-44d8-adf4-5a12279f2a38 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:23.434689 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784a210-d4cb-44d8-adf4-5a12279f2a38 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:23.434964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784a210-d4cb-44d8-adf4-5a12279f2a38 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:23.436344 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 72/156] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:23.441507 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784a210-d4cb-44d8-adf4-5a12279f2a38 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:23.505194 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb404e-bab8-4e7c-bf17-b007bfc45462 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:23.511055 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:23.545747 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24888e59-9cc3-475a-88a1-4d45817677d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:23.546383 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24888e59-9cc3-475a-88a1-4d45817677d1 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:23.546660 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24888e59-9cc3-475a-88a1-4d45817677d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:23.546923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24888e59-9cc3-475a-88a1-4d45817677d1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:23.561162 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24888e59-9cc3-475a-88a1-4d45817677d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:23.567758 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-24888e59-9cc3-475a-88a1-4d45817677d1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:46:23.567758 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:23.567758 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:23.567758 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:23.567758 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:23.567758 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:23.567758 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:23.567758 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:23.567758 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:23.567758 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:23.567758 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:23.567758 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:23.567758 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:23.567758 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:23.567758 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:23.567758 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:23.568705 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:23.568705 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:23.568705 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:23.568705 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:23.568705 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:23.568705 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:23.568705 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:23.568705 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:23.568705 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:23.568705 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:23.568705 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:23.568705 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:23.568705 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:23.568705 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:23.568705 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:23.568705 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:23.569475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:23.569475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:23.569475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:23.569475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:23.569475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:23.569475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:23.569475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:23.569475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:23.569475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:23.569475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:23.569475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:46:23.569475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:46:23.569475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 85/157] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:46:23.570588 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a63b8-abea-4e7a-8330-7aec7166b553 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:23.570588 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a63b8-abea-4e7a-8330-7aec7166b553 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:23.570881 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a63b8-abea-4e7a-8330-7aec7166b553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:23.585180 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784a210-d4cb-44d8-adf4-5a12279f2a38 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:23.595599 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-226a63b8-abea-4e7a-8330-7aec7166b553 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:23.596923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-226a63b8-abea-4e7a-8330-7aec7166b553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:23.609029 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:23.623860 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4a1b5-c9e7-47eb-a054-56d58d11538c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:23.625721 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4a1b5-c9e7-47eb-a054-56d58d11538c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:23.625721 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4a1b5-c9e7-47eb-a054-56d58d11538c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:23.625721 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4a1b5-c9e7-47eb-a054-56d58d11538c None admin] PATH_INFO: `/v3/users/2bca44a120f84b35b219ce7bed5a328a` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:23.626680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f4a1b5-c9e7-47eb-a054-56d58d11538c None admin] RBAC: Authorizing `identity:get_user(user_id=2bca44a120f84b35b219ce7bed5a328a)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:23.628376 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f4a1b5-c9e7-47eb-a054-56d58d11538c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:23.630170 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 73/158] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:23 2022] GET /identity/v3/users/2bca44a120f84b35b219ce7bed5a328a => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:46:23.632270 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4a1b5-c9e7-47eb-a054-56d58d11538c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:23.679473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 86/159] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:23.680372 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a63b8-abea-4e7a-8330-7aec7166b553 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:23.742274 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4a1b5-c9e7-47eb-a054-56d58d11538c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:23.761114 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:23.793349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952c0ba-ea74-4be2-82bf-12152ccfefee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:23.793980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952c0ba-ea74-4be2-82bf-12152ccfefee None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:23.794260 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952c0ba-ea74-4be2-82bf-12152ccfefee None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:23.794532 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952c0ba-ea74-4be2-82bf-12152ccfefee None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:23.795211 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1952c0ba-ea74-4be2-82bf-12152ccfefee None admin] RBAC: Adding query filter params (name=reader) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:23.795515 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1952c0ba-ea74-4be2-82bf-12152ccfefee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:23.796946 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1952c0ba-ea74-4be2-82bf-12152ccfefee None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:23.814488 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 74/160] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Sep 9 20:46:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:23.821975 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952c0ba-ea74-4be2-82bf-12152ccfefee None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:23.824901 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a63b8-abea-4e7a-8330-7aec7166b553 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:23.884553 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8793104-fe22-4746-a151-500da796ff49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:23.885714 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8793104-fe22-4746-a151-500da796ff49 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:23.886194 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8793104-fe22-4746-a151-500da796ff49 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:23.887129 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8793104-fe22-4746-a151-500da796ff49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:23.907633 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:23.975721 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8793104-fe22-4746-a151-500da796ff49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:23.977291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8793104-fe22-4746-a151-500da796ff49 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:24.014774 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952c0ba-ea74-4be2-82bf-12152ccfefee None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:24.055235 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2038c-3353-4007-9fd3-6d3cfa3a5ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:24.056123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2038c-3353-4007-9fd3-6d3cfa3a5ef2 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:24.056510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2038c-3353-4007-9fd3-6d3cfa3a5ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:24.056887 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2038c-3353-4007-9fd3-6d3cfa3a5ef2 None admin] PATH_INFO: `/v3/users/57f1e4c0e3bb46ee9871ed96d96328fa` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:24.058146 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e2038c-3353-4007-9fd3-6d3cfa3a5ef2 None admin] RBAC: Authorizing `identity:get_user(user_id=57f1e4c0e3bb46ee9871ed96d96328fa)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:24.060205 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e2038c-3353-4007-9fd3-6d3cfa3a5ef2 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:24.062498 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 75/161] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:23 2022] GET /identity/v3/users/57f1e4c0e3bb46ee9871ed96d96328fa => generated 289 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:24.063138 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7768e0-5d1d-45c0-bcab-cf59b5a1b169 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:24.063606 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7768e0-5d1d-45c0-bcab-cf59b5a1b169 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:24.063953 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7768e0-5d1d-45c0-bcab-cf59b5a1b169 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:24.065533 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 76/162] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:24.068935 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7768e0-5d1d-45c0-bcab-cf59b5a1b169 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:24.098786 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 87/163] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:46:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:46:24.105585 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b56a50-a861-498f-bf8b-9857e850798c None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:24.106048 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b56a50-a861-498f-bf8b-9857e850798c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:24.106497 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b56a50-a861-498f-bf8b-9857e850798c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:24.126237 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:24.128884 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-23b56a50-a861-498f-bf8b-9857e850798c None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:24.130275 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-23b56a50-a861-498f-bf8b-9857e850798c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:24.181518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:24.189037 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7768e0-5d1d-45c0-bcab-cf59b5a1b169 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:24.228233 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 88/164] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:24.228233 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b56a50-a861-498f-bf8b-9857e850798c None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:24.248603 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eae562-8d75-438f-ae5d-e918841d40e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:24.249257 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eae562-8d75-438f-ae5d-e918841d40e8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:24.249541 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eae562-8d75-438f-ae5d-e918841d40e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:24.249820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eae562-8d75-438f-ae5d-e918841d40e8 None admin] PATH_INFO: `/v3/projects/1464b3dd76734fb593f3ed3582957e86` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:24.252714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eae562-8d75-438f-ae5d-e918841d40e8 None admin] RBAC: Authorizing `identity:get_project(project_id=1464b3dd76734fb593f3ed3582957e86)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:24.254201 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eae562-8d75-438f-ae5d-e918841d40e8 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:24.257437 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 77/165] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:24 2022] GET /identity/v3/projects/1464b3dd76734fb593f3ed3582957e86 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:24.258198 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eae562-8d75-438f-ae5d-e918841d40e8 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:24.456672 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b56a50-a861-498f-bf8b-9857e850798c None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:24.499382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eae562-8d75-438f-ae5d-e918841d40e8 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:24.526404 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:24.535728 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731171c-a939-4a1b-8bff-82dc6092f2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:24.536407 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731171c-a939-4a1b-8bff-82dc6092f2cb None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:24.536684 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731171c-a939-4a1b-8bff-82dc6092f2cb None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:24.538930 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731171c-a939-4a1b-8bff-82dc6092f2cb None admin] PATH_INFO: `/v3/projects/1464b3dd76734fb593f3ed3582957e86` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:24.539021 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731171c-a939-4a1b-8bff-82dc6092f2cb None admin] RBAC: Authorizing `identity:get_project(project_id=1464b3dd76734fb593f3ed3582957e86)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:24.540542 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731171c-a939-4a1b-8bff-82dc6092f2cb None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:24.542187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 89/166] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:24 2022] GET /identity/v3/projects/1464b3dd76734fb593f3ed3582957e86 => generated 298 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:24.542932 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731171c-a939-4a1b-8bff-82dc6092f2cb None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:24.561667 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0866441-acd5-4c53-a51a-c19bf23c93c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:24.562748 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0866441-acd5-4c53-a51a-c19bf23c93c2 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:24.563224 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0866441-acd5-4c53-a51a-c19bf23c93c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:24.563788 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0866441-acd5-4c53-a51a-c19bf23c93c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:24.583492 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0866441-acd5-4c53-a51a-c19bf23c93c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:24.589615 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:24.592954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-e0866441-acd5-4c53-a51a-c19bf23c93c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:46:24.592954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:24.592954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:24.592954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:24.592954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:24.592954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:24.592954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:24.592954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:24.592954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:24.592954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:24.592954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:24.592954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:24.592954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:24.592954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:24.592954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:24.592954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:24.593832 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:24.593832 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:24.593832 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:24.593832 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:24.593832 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:24.593832 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:24.593832 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:24.593832 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:24.593832 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:24.593832 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:24.593832 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:24.593832 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:24.593832 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:24.593832 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:24.593832 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:24.593832 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:24.594786 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:24.594786 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:24.594786 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:24.594786 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:24.594786 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:24.594786 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:24.594786 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:24.594786 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:24.594786 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:24.594786 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:24.594786 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:46:24.594786 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:46:24.594786 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 78/167] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:46:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:24.597592 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0866441-acd5-4c53-a51a-c19bf23c93c2 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:24.672375 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731171c-a939-4a1b-8bff-82dc6092f2cb None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:24.736817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e813c-acb6-4365-9cbe-1a81fbdc3fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:24.737931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e813c-acb6-4365-9cbe-1a81fbdc3fb8 None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:24.738399 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e813c-acb6-4365-9cbe-1a81fbdc3fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:24.739553 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e813c-acb6-4365-9cbe-1a81fbdc3fb8 None admin] PATH_INFO: `/v3/projects/1464b3dd76734fb593f3ed3582957e86/users/57f1e4c0e3bb46ee9871ed96d96328fa/roles/e01870d6deef4f1ebbc7470c342046a6` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:24.743683 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e813c-acb6-4365-9cbe-1a81fbdc3fb8 None admin] RBAC: Authorizing `identity:create_grant(project_id=1464b3dd76734fb593f3ed3582957e86, user_id=57f1e4c0e3bb46ee9871ed96d96328fa, role_id=e01870d6deef4f1ebbc7470c342046a6)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:24.746519 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e813c-acb6-4365-9cbe-1a81fbdc3fb8 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:24.768823 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0866441-acd5-4c53-a51a-c19bf23c93c2 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:24.803332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c160b5-fb76-41ba-93fe-6362ee9e6683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:24.803955 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c160b5-fb76-41ba-93fe-6362ee9e6683 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:24.804284 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c160b5-fb76-41ba-93fe-6362ee9e6683 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:24.804553 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c160b5-fb76-41ba-93fe-6362ee9e6683 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:24.805357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c160b5-fb76-41ba-93fe-6362ee9e6683 None admin] RBAC: Adding query filter params (role.id=13072ad4b81649459f59b8137db492ec, scope.project.id=1464b3dd76734fb593f3ed3582957e86, user.id=2bca44a120f84b35b219ce7bed5a328a) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:24.806211 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c160b5-fb76-41ba-93fe-6362ee9e6683 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:24.807739 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c160b5-fb76-41ba-93fe-6362ee9e6683 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:24.814326 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 79/168] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:46:24 2022] GET /identity/v3/role_assignments?user.id=2bca44a120f84b35b219ce7bed5a328a&scope.project.id=1464b3dd76734fb593f3ed3582957e86&role.id=13072ad4b81649459f59b8137db492ec => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:24.815244 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c160b5-fb76-41ba-93fe-6362ee9e6683 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:24.823723 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 90/169] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Sep 9 20:46:24 2022] PUT /identity/v3/projects/1464b3dd76734fb593f3ed3582957e86/users/57f1e4c0e3bb46ee9871ed96d96328fa/roles/e01870d6deef4f1ebbc7470c342046a6 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:46:24.825353 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e813c-acb6-4365-9cbe-1a81fbdc3fb8 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:24.892494 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:24.960559 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e813c-acb6-4365-9cbe-1a81fbdc3fb8 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:24.964545 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:25.025792 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c160b5-fb76-41ba-93fe-6362ee9e6683 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:25.028197 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086e48d-ebc2-4318-9a50-ae4eee6f5da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:25.028830 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086e48d-ebc2-4318-9a50-ae4eee6f5da4 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:25.029108 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086e48d-ebc2-4318-9a50-ae4eee6f5da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:25.029379 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086e48d-ebc2-4318-9a50-ae4eee6f5da4 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:25.030071 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086e48d-ebc2-4318-9a50-ae4eee6f5da4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:25.030377 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086e48d-ebc2-4318-9a50-ae4eee6f5da4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:25.031824 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086e48d-ebc2-4318-9a50-ae4eee6f5da4 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:25.051512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 91/170] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:25.057594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086e48d-ebc2-4318-9a50-ae4eee6f5da4 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:25.104519 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc223102-f1d6-4163-a453-874ef7795b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:25.105183 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc223102-f1d6-4163-a453-874ef7795b8a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:25.105466 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc223102-f1d6-4163-a453-874ef7795b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:25.105743 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc223102-f1d6-4163-a453-874ef7795b8a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:25.127277 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc223102-f1d6-4163-a453-874ef7795b8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:25.135353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-bc223102-f1d6-4163-a453-874ef7795b8a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:25.135353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:25.135353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:25.135353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:25.135353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:25.135353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:25.135353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:25.135353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:25.135353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:25.135353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:25.135353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:25.135353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:25.135353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:25.135353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:25.135353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:25.135353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:25.136335 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:25.136335 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:25.136335 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:25.136335 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:25.136335 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:25.136335 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:25.136335 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:25.136335 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:25.136335 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:25.136335 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:25.136335 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:25.136335 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:25.136335 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:25.136335 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:25.136335 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:25.136335 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:25.137144 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:25.137144 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:25.137144 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:25.137144 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:25.137144 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:25.137144 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:25.137144 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:25.137144 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:25.137144 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:25.137144 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:25.137144 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:25.137144 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:46:25.143383 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 80/171] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:25.148573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc223102-f1d6-4163-a453-874ef7795b8a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:25.257588 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086e48d-ebc2-4318-9a50-ae4eee6f5da4 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:25.327025 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39931309-dad6-4fd7-accf-52e708052429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:25.327590 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39931309-dad6-4fd7-accf-52e708052429 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:25.327875 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39931309-dad6-4fd7-accf-52e708052429 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:25.328163 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39931309-dad6-4fd7-accf-52e708052429 None admin] PATH_INFO: `/v3/users/2e3a7a2557844cc78377f788a7c8c3ae` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:25.329974 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39931309-dad6-4fd7-accf-52e708052429 None admin] RBAC: Authorizing `identity:get_user(user_id=2e3a7a2557844cc78377f788a7c8c3ae)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:25.331436 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39931309-dad6-4fd7-accf-52e708052429 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:25.333065 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 92/172] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:25 2022] GET /identity/v3/users/2e3a7a2557844cc78377f788a7c8c3ae => generated 247 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:46:25.339215 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39931309-dad6-4fd7-accf-52e708052429 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:25.342123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:25.353019 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc223102-f1d6-4163-a453-874ef7795b8a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:25.398397 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:25.416268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83f184-88e5-4fb8-be4d-a463eaf6cf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:25.416888 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83f184-88e5-4fb8-be4d-a463eaf6cf58 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:25.417170 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83f184-88e5-4fb8-be4d-a463eaf6cf58 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:25.417440 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83f184-88e5-4fb8-be4d-a463eaf6cf58 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:25.418114 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c83f184-88e5-4fb8-be4d-a463eaf6cf58 None admin] RBAC: Adding query filter params (name=member) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:25.418421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c83f184-88e5-4fb8-be4d-a463eaf6cf58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:25.419819 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c83f184-88e5-4fb8-be4d-a463eaf6cf58 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:25.434685 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 81/173] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Sep 9 20:46:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:25.440527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83f184-88e5-4fb8-be4d-a463eaf6cf58 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:25.535156 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39931309-dad6-4fd7-accf-52e708052429 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:25.606212 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393a017-4975-4ab0-967e-dae407a76b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:25.606814 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393a017-4975-4ab0-967e-dae407a76b88 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:25.607037 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393a017-4975-4ab0-967e-dae407a76b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:25.607276 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393a017-4975-4ab0-967e-dae407a76b88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:25.626573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83f184-88e5-4fb8-be4d-a463eaf6cf58 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:25.690359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe7ed3f-9d11-4af9-b77d-6afcdbfff0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:25.691004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7ed3f-9d11-4af9-b77d-6afcdbfff0d7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:25.691286 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7ed3f-9d11-4af9-b77d-6afcdbfff0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:25.691560 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7ed3f-9d11-4af9-b77d-6afcdbfff0d7 None admin] PATH_INFO: `/v3/users/2bca44a120f84b35b219ce7bed5a328a` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:25.693104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe7ed3f-9d11-4af9-b77d-6afcdbfff0d7 None admin] RBAC: Authorizing `identity:get_user(user_id=2bca44a120f84b35b219ce7bed5a328a)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:25.697421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe7ed3f-9d11-4af9-b77d-6afcdbfff0d7 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:25.699457 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 82/174] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:25 2022] GET /identity/v3/users/2bca44a120f84b35b219ce7bed5a328a => generated 275 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:25.704861 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe7ed3f-9d11-4af9-b77d-6afcdbfff0d7 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:25.737226 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393a017-4975-4ab0-967e-dae407a76b88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:25.738705 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393a017-4975-4ab0-967e-dae407a76b88 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:25.764243 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:25.808434 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:25.899141 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe7ed3f-9d11-4af9-b77d-6afcdbfff0d7 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:25.902309 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 93/175] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:46:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 20:46:25.908524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393a017-4975-4ab0-967e-dae407a76b88 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:25.975614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e725a5aa-4a9a-491c-a860-b99cad8a1605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:25.976641 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725a5aa-4a9a-491c-a860-b99cad8a1605 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:25.977144 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725a5aa-4a9a-491c-a860-b99cad8a1605 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:25.978160 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725a5aa-4a9a-491c-a860-b99cad8a1605 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:26.108797 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393a017-4975-4ab0-967e-dae407a76b88 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:26.118706 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e725a5aa-4a9a-491c-a860-b99cad8a1605 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:26.120240 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e725a5aa-4a9a-491c-a860-b99cad8a1605 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:26.163686 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:26.167268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:26.171238 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfacb72-806f-4f95-af98-e8d80ee8aef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:26.171887 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfacb72-806f-4f95-af98-e8d80ee8aef7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:26.172191 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfacb72-806f-4f95-af98-e8d80ee8aef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:26.172461 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfacb72-806f-4f95-af98-e8d80ee8aef7 None admin] PATH_INFO: `/v3/projects/1464b3dd76734fb593f3ed3582957e86` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:26.174275 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfacb72-806f-4f95-af98-e8d80ee8aef7 None admin] RBAC: Authorizing `identity:get_project(project_id=1464b3dd76734fb593f3ed3582957e86)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:26.175698 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfacb72-806f-4f95-af98-e8d80ee8aef7 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:26.180713 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 94/176] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:25 2022] GET /identity/v3/projects/1464b3dd76734fb593f3ed3582957e86 => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:46:26.185298 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfacb72-806f-4f95-af98-e8d80ee8aef7 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:26.288021 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 83/177] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:46:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 20:46:26.289314 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826ed98-19c7-4fcc-9f0c-fe3c7ff4a49b None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:26.289600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826ed98-19c7-4fcc-9f0c-fe3c7ff4a49b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:26.290447 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826ed98-19c7-4fcc-9f0c-fe3c7ff4a49b None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:26.291816 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 84/178] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:26.295476 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826ed98-19c7-4fcc-9f0c-fe3c7ff4a49b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:26.373782 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfacb72-806f-4f95-af98-e8d80ee8aef7 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:26.423810 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826ed98-19c7-4fcc-9f0c-fe3c7ff4a49b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:26.456191 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:26.465480 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319b2d7-610c-4004-be92-3d100b8bff25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:26.466110 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319b2d7-610c-4004-be92-3d100b8bff25 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:26.466391 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319b2d7-610c-4004-be92-3d100b8bff25 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:26.466661 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319b2d7-610c-4004-be92-3d100b8bff25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:26.469249 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea603a3e-2c20-4f35-ba5b-b9a1339d3bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:26.469362 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c319b2d7-610c-4004-be92-3d100b8bff25 None admin] RBAC: Adding query filter params (role.id=fcbf519fbc84458f913671a5a1d0914c, scope.project.id=1464b3dd76734fb593f3ed3582957e86, user.id=2e3a7a2557844cc78377f788a7c8c3ae) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:26.469362 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c319b2d7-610c-4004-be92-3d100b8bff25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:26.469494 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea603a3e-2c20-4f35-ba5b-b9a1339d3bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:26.469494 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea603a3e-2c20-4f35-ba5b-b9a1339d3bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:26.469494 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea603a3e-2c20-4f35-ba5b-b9a1339d3bf5 None admin] PATH_INFO: `/v3/projects/ae54b4a48e9f41c0a6d977e47550193f` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:26.469681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c319b2d7-610c-4004-be92-3d100b8bff25 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:26.470308 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea603a3e-2c20-4f35-ba5b-b9a1339d3bf5 None admin] RBAC: Authorizing `identity:get_project(project_id=ae54b4a48e9f41c0a6d977e47550193f)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:26.472451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea603a3e-2c20-4f35-ba5b-b9a1339d3bf5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:26.474302 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 85/179] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:26 2022] GET /identity/v3/projects/ae54b4a48e9f41c0a6d977e47550193f => generated 312 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:26.475600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27443d-8c48-43c4-a028-951c1099300d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:26.476100 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27443d-8c48-43c4-a028-951c1099300d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:26.476581 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27443d-8c48-43c4-a028-951c1099300d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:26.483763 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 95/180] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:46:26 2022] GET /identity/v3/role_assignments?user.id=2e3a7a2557844cc78377f788a7c8c3ae&scope.project.id=1464b3dd76734fb593f3ed3582957e86&role.id=fcbf519fbc84458f913671a5a1d0914c => generated 608 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:26.484634 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319b2d7-610c-4004-be92-3d100b8bff25 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:26.498799 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-2f27443d-8c48-43c4-a028-951c1099300d None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:26.500386 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2f27443d-8c48-43c4-a028-951c1099300d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:26.637136 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 86/181] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:26.642078 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:26.691743 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319b2d7-610c-4004-be92-3d100b8bff25 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:26.730526 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076d325-71eb-4dca-a327-5ca43ebfc57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:26.731494 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076d325-71eb-4dca-a327-5ca43ebfc57b None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:26.731975 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076d325-71eb-4dca-a327-5ca43ebfc57b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:26.732503 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076d325-71eb-4dca-a327-5ca43ebfc57b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:26.733440 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076d325-71eb-4dca-a327-5ca43ebfc57b None admin] RBAC: Adding query filter params (role.id=c59d891d042e48bab625f7b3666ef536, scope.project.id=ae54b4a48e9f41c0a6d977e47550193f, user.id=2bca44a120f84b35b219ce7bed5a328a) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:26.733974 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076d325-71eb-4dca-a327-5ca43ebfc57b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:26.735609 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076d325-71eb-4dca-a327-5ca43ebfc57b None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:26.742306 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 96/182] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:46:26 2022] GET /identity/v3/role_assignments?user.id=2bca44a120f84b35b219ce7bed5a328a&scope.project.id=ae54b4a48e9f41c0a6d977e47550193f&role.id=c59d891d042e48bab625f7b3666ef536 => generated 608 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:27.011625 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f27443d-8c48-43c4-a028-951c1099300d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:27.080419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f9b7b-18ee-4145-baf8-55d945625556 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:27.081029 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f9b7b-18ee-4145-baf8-55d945625556 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:27.081502 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f9b7b-18ee-4145-baf8-55d945625556 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:27.083175 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 97/183] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:46:27.089059 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42eaed1-8be8-4b1c-8c80-2cfed05fb436 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:27.089059 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42eaed1-8be8-4b1c-8c80-2cfed05fb436 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:27.089402 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42eaed1-8be8-4b1c-8c80-2cfed05fb436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:27.125702 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-d42eaed1-8be8-4b1c-8c80-2cfed05fb436 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:27.127032 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d42eaed1-8be8-4b1c-8c80-2cfed05fb436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:27.140933 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f27443d-8c48-43c4-a028-951c1099300d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:27.185105 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:27.197085 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b8e2d1-ed8f-452f-ad2b-ebc90da04816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:27.197734 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8e2d1-ed8f-452f-ad2b-ebc90da04816 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:27.198014 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8e2d1-ed8f-452f-ad2b-ebc90da04816 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:27.198287 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8e2d1-ed8f-452f-ad2b-ebc90da04816 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:27.209596 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b8e2d1-ed8f-452f-ad2b-ebc90da04816 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:27.215139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-40b8e2d1-ed8f-452f-ad2b-ebc90da04816 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:46:27.215139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:27.215139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:27.215139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:27.215139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:27.215139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:27.215139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:27.215139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:27.215139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:27.215139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:27.215139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:27.215139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:27.215139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:27.215139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:27.215139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:27.215139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:27.216452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:27.216452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:27.216452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:27.216452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:27.216452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:27.216452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:27.216452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:27.216452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:27.216452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:27.216452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:27.216452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:27.216452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:27.216452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:27.216452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:27.216452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:27.216452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:27.217441 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:27.217441 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:27.217441 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:27.217441 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:27.217441 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:27.217441 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:27.217441 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:27.217441 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:27.217441 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:27.217441 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:27.217441 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:46:27.217441 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:46:27.219386 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 87/184] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:46:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:27.225849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b8e2d1-ed8f-452f-ad2b-ebc90da04816 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:27.230129 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 98/185] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:46:27.343553 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b8e2d1-ed8f-452f-ad2b-ebc90da04816 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:27.354393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94310a40-d58c-4c75-93d9-0ada58aedab7 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:27.354682 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94310a40-d58c-4c75-93d9-0ada58aedab7 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:27.354959 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94310a40-d58c-4c75-93d9-0ada58aedab7 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:27.356488 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 99/186] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:27.362551 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab80d0-c992-4a75-b456-17606b6398f1 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:27.362551 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab80d0-c992-4a75-b456-17606b6398f1 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:27.362551 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab80d0-c992-4a75-b456-17606b6398f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:27.385322 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-0aab80d0-c992-4a75-b456-17606b6398f1 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:27.386920 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0aab80d0-c992-4a75-b456-17606b6398f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:27.392756 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7375333-15d3-45d1-a4b5-f38dd991dc9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:27.392906 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7375333-15d3-45d1-a4b5-f38dd991dc9a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:27.393190 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7375333-15d3-45d1-a4b5-f38dd991dc9a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:27.393466 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7375333-15d3-45d1-a4b5-f38dd991dc9a None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:27.394148 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7375333-15d3-45d1-a4b5-f38dd991dc9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:27.394454 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7375333-15d3-45d1-a4b5-f38dd991dc9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:27.396009 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7375333-15d3-45d1-a4b5-f38dd991dc9a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:27.405634 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 88/187] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:27.411825 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7375333-15d3-45d1-a4b5-f38dd991dc9a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:27.453505 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:27.467419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 100/188] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:27.475111 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:27.527641 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7375333-15d3-45d1-a4b5-f38dd991dc9a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:27.537451 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab80d0-c992-4a75-b456-17606b6398f1 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:27.569979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc808921-553d-4c40-a60b-9cfc466aa8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:27.570926 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc808921-553d-4c40-a60b-9cfc466aa8b4 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:27.571629 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc808921-553d-4c40-a60b-9cfc466aa8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:27.571916 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc808921-553d-4c40-a60b-9cfc466aa8b4 None admin] PATH_INFO: `/v3/users/fcafeca38d4942cf9f052bd972453739` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:27.586859 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc808921-553d-4c40-a60b-9cfc466aa8b4 None admin] RBAC: Authorizing `identity:get_user(user_id=fcafeca38d4942cf9f052bd972453739)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:27.588368 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc808921-553d-4c40-a60b-9cfc466aa8b4 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:27.590029 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 89/189] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:27 2022] GET /identity/v3/users/fcafeca38d4942cf9f052bd972453739 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:27.595806 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc808921-553d-4c40-a60b-9cfc466aa8b4 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:27.662174 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab80d0-c992-4a75-b456-17606b6398f1 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:27.698879 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc808921-553d-4c40-a60b-9cfc466aa8b4 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:27.702000 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b973441-169f-4bda-89c0-f115e8eb2b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:27.702639 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b973441-169f-4bda-89c0-f115e8eb2b8b None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:27.702973 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b973441-169f-4bda-89c0-f115e8eb2b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:27.703246 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b973441-169f-4bda-89c0-f115e8eb2b8b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:27.707095 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:27.714533 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b973441-169f-4bda-89c0-f115e8eb2b8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:27.720460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-5b973441-169f-4bda-89c0-f115e8eb2b8b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:46:27.720460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:27.720460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:27.720460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:27.720460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:27.720460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:27.720460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:27.720460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:27.720460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:27.720460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:27.720460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:27.720460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:27.720460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:27.720460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:27.720460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:27.720460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:27.721345 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:27.721345 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:27.721345 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:27.721345 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:27.721345 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:27.721345 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:27.721345 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:27.721345 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:27.721345 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:27.721345 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:27.721345 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:27.721345 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:27.721345 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:27.721345 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:27.721345 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:27.721345 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:27.723190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:27.723190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:27.723190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:27.723190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:27.723190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:27.723190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:27.723190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:27.723190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:27.723190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:27.723190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:27.723190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:46:27.723190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:46:27.723190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 101/190] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Sep 9 20:46:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:46:27.728513 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b973441-169f-4bda-89c0-f115e8eb2b8b None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:27.740604 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a8cd3-8e9d-4872-901f-eb3faf45399d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:27.741273 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a8cd3-8e9d-4872-901f-eb3faf45399d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:27.741556 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a8cd3-8e9d-4872-901f-eb3faf45399d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:27.741829 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a8cd3-8e9d-4872-901f-eb3faf45399d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:27.784746 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:27.816989 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a8cd3-8e9d-4872-901f-eb3faf45399d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:27.818734 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a8cd3-8e9d-4872-901f-eb3faf45399d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:27.844422 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b973441-169f-4bda-89c0-f115e8eb2b8b None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:27.880046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf227bd-3b59-417f-b885-c8c9ab7a4934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:27.880853 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf227bd-3b59-417f-b885-c8c9ab7a4934 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:27.881010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf227bd-3b59-417f-b885-c8c9ab7a4934 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:27.881280 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf227bd-3b59-417f-b885-c8c9ab7a4934 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:27.882132 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf227bd-3b59-417f-b885-c8c9ab7a4934 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:27.882237 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf227bd-3b59-417f-b885-c8c9ab7a4934 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:27.883620 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf227bd-3b59-417f-b885-c8c9ab7a4934 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:27.892293 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 102/191] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:46:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:27.892434 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf227bd-3b59-417f-b885-c8c9ab7a4934 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:27.927717 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 90/192] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:46:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 20:46:27.931850 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a8cd3-8e9d-4872-901f-eb3faf45399d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:27.940730 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:27.996541 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf227bd-3b59-417f-b885-c8c9ab7a4934 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:28.006690 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:28.031346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a8cd3-8e9d-4872-901f-eb3faf45399d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:28.032513 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc01de6-2d69-4720-a9f2-0273a338e79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:28.033318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc01de6-2d69-4720-a9f2-0273a338e79d None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:28.033641 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc01de6-2d69-4720-a9f2-0273a338e79d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:28.033911 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc01de6-2d69-4720-a9f2-0273a338e79d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:28.035007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc01de6-2d69-4720-a9f2-0273a338e79d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:28.036479 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc01de6-2d69-4720-a9f2-0273a338e79d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:28.038032 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 103/193] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:46:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:28.039238 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d24e2-35a6-4153-8807-afc4442af9f8 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:28.039729 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d24e2-35a6-4153-8807-afc4442af9f8 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:28.040316 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d24e2-35a6-4153-8807-afc4442af9f8 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:28.044452 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 104/194] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:28.045603 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d24e2-35a6-4153-8807-afc4442af9f8 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:28.067145 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc5a56-3d38-4ba7-921d-1ed198de5b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:28.067777 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc5a56-3d38-4ba7-921d-1ed198de5b74 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:28.068054 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc5a56-3d38-4ba7-921d-1ed198de5b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:28.068368 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc5a56-3d38-4ba7-921d-1ed198de5b74 None admin] PATH_INFO: `/v3/users/fcafeca38d4942cf9f052bd972453739` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:28.069478 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc5a56-3d38-4ba7-921d-1ed198de5b74 None admin] RBAC: Authorizing `identity:get_user(user_id=fcafeca38d4942cf9f052bd972453739)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:28.070930 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc5a56-3d38-4ba7-921d-1ed198de5b74 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:28.073036 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 91/195] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:27 2022] GET /identity/v3/users/fcafeca38d4942cf9f052bd972453739 => generated 283 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:28.073764 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc5a56-3d38-4ba7-921d-1ed198de5b74 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:28.145577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:28.151814 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d24e2-35a6-4153-8807-afc4442af9f8 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:28.170275 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc5a56-3d38-4ba7-921d-1ed198de5b74 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:28.183909 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9f2c6-6e1d-4e5d-8b0c-8371679cda77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:28.184929 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9f2c6-6e1d-4e5d-8b0c-8371679cda77 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:28.185212 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9f2c6-6e1d-4e5d-8b0c-8371679cda77 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:28.185485 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9f2c6-6e1d-4e5d-8b0c-8371679cda77 None admin] PATH_INFO: `/v3/projects/e13bca418ede487ab7d1230e28403173` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:28.186606 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f9f2c6-6e1d-4e5d-8b0c-8371679cda77 None admin] RBAC: Authorizing `identity:get_project(project_id=e13bca418ede487ab7d1230e28403173)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:28.188091 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f9f2c6-6e1d-4e5d-8b0c-8371679cda77 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:28.189738 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 105/196] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:28 2022] GET /identity/v3/projects/e13bca418ede487ab7d1230e28403173 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:28.194823 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f0778-19bb-404b-9e78-5a22f3b52ae2 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:28.194823 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f0778-19bb-404b-9e78-5a22f3b52ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:28.194823 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f0778-19bb-404b-9e78-5a22f3b52ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:28.204808 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadcae3-ed7c-4fa2-a652-455213bc7847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:28.205413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadcae3-ed7c-4fa2-a652-455213bc7847 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:28.206058 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadcae3-ed7c-4fa2-a652-455213bc7847 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:28.206058 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadcae3-ed7c-4fa2-a652-455213bc7847 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:28.206661 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aadcae3-ed7c-4fa2-a652-455213bc7847 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:28.208262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aadcae3-ed7c-4fa2-a652-455213bc7847 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:28.212580 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-327f0778-19bb-404b-9e78-5a22f3b52ae2 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:28.214162 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-327f0778-19bb-404b-9e78-5a22f3b52ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:28.258185 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:28.281947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 106/197] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:28.282745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f0778-19bb-404b-9e78-5a22f3b52ae2 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:28.375148 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f0778-19bb-404b-9e78-5a22f3b52ae2 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:28.406778 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c3616-a289-4c3e-88e3-11c662c914a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:28.407404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c3616-a289-4c3e-88e3-11c662c914a7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:28.407683 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c3616-a289-4c3e-88e3-11c662c914a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:28.407954 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c3616-a289-4c3e-88e3-11c662c914a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:28.470607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c3616-a289-4c3e-88e3-11c662c914a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:28.472052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c3616-a289-4c3e-88e3-11c662c914a7 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:28.566963 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 107/198] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:46:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:46:28.567906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c3616-a289-4c3e-88e3-11c662c914a7 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:28.571701 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 92/199] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Sep 9 20:46:28 2022] POST /identity/v3/users => generated 297 bytes in 498 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:46:28.576240 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadcae3-ed7c-4fa2-a652-455213bc7847 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:28.585332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:28.660722 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:28.691880 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c3616-a289-4c3e-88e3-11c662c914a7 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:28.692733 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadcae3-ed7c-4fa2-a652-455213bc7847 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:28.725813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:28.730039 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eeb46f-949c-46e0-9774-b3e799b84a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:28.730659 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eeb46f-949c-46e0-9774-b3e799b84a98 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:28.730939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eeb46f-949c-46e0-9774-b3e799b84a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:28.731212 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eeb46f-949c-46e0-9774-b3e799b84a98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:28.731950 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eeb46f-949c-46e0-9774-b3e799b84a98 None admin] RBAC: Adding query filter params (role.id=fcbf519fbc84458f913671a5a1d0914c, scope.project.id=e13bca418ede487ab7d1230e28403173, user.id=fcafeca38d4942cf9f052bd972453739) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:28.732300 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eeb46f-949c-46e0-9774-b3e799b84a98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:28.732910 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c442ad1-e876-432f-aea1-ad9f11567454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:28.733565 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c442ad1-e876-432f-aea1-ad9f11567454 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:28.733741 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eeb46f-949c-46e0-9774-b3e799b84a98 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:28.733839 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c442ad1-e876-432f-aea1-ad9f11567454 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:28.734112 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c442ad1-e876-432f-aea1-ad9f11567454 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:28.739845 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 108/200] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:46:28 2022] GET /identity/v3/role_assignments?user.id=fcafeca38d4942cf9f052bd972453739&scope.project.id=e13bca418ede487ab7d1230e28403173&role.id=fcbf519fbc84458f913671a5a1d0914c => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:28.749200 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eeb46f-949c-46e0-9774-b3e799b84a98 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:28.751421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c442ad1-e876-432f-aea1-ad9f11567454 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:28.756761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-1c442ad1-e876-432f-aea1-ad9f11567454 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:46:28.756761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:28.756761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:28.756761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:28.756761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:28.756761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:28.756761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:28.756761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:28.756761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:28.756761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:28.756761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:28.756761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:28.756761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:28.756761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:28.756761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:28.756761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:28.757800 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:28.757800 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:28.757800 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:28.757800 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:28.757800 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:28.757800 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:28.757800 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:28.757800 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:28.757800 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:28.757800 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:28.757800 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:28.757800 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:28.757800 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:28.757800 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:28.757800 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:28.757800 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:28.759168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:28.759168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:28.759168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:28.759168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:28.759168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:28.759168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:28.759168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:28.759168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:28.759168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:28.759168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:28.759168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:46:28.759168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:46:28.759168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 93/201] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:28.766865 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c442ad1-e876-432f-aea1-ad9f11567454 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:28.891646 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eeb46f-949c-46e0-9774-b3e799b84a98 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:28.916185 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c442ad1-e876-432f-aea1-ad9f11567454 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:28.946616 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a05152-8157-403d-9953-5176be6c06cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:28.947531 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a05152-8157-403d-9953-5176be6c06cf None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:28.947982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a05152-8157-403d-9953-5176be6c06cf None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:28.948282 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a05152-8157-403d-9953-5176be6c06cf None admin] PATH_INFO: `/v3/projects/e13bca418ede487ab7d1230e28403173` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:28.950336 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a05152-8157-403d-9953-5176be6c06cf None admin] RBAC: Authorizing `identity:get_project(project_id=e13bca418ede487ab7d1230e28403173)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:28.951797 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a05152-8157-403d-9953-5176be6c06cf None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:28.953642 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 109/202] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:28 2022] GET /identity/v3/projects/e13bca418ede487ab7d1230e28403173 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:28.959922 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b982833-a8bc-423b-b6f7-526412488b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:28.960913 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b982833-a8bc-423b-b6f7-526412488b5f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:28.961049 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a05152-8157-403d-9953-5176be6c06cf None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:28.961221 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b982833-a8bc-423b-b6f7-526412488b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:28.961478 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b982833-a8bc-423b-b6f7-526412488b5f None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:28.962727 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b982833-a8bc-423b-b6f7-526412488b5f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:28.963039 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b982833-a8bc-423b-b6f7-526412488b5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:28.964454 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b982833-a8bc-423b-b6f7-526412488b5f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:28.976285 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 94/203] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Sep 9 20:46:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:28.976764 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:28.982315 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b982833-a8bc-423b-b6f7-526412488b5f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:29.056604 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:29.106451 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a05152-8157-403d-9953-5176be6c06cf None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:29.106721 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b982833-a8bc-423b-b6f7-526412488b5f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:29.146392 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325d899-cd80-46e1-9892-976fc02a2be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:29.147350 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325d899-cd80-46e1-9892-976fc02a2be7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:29.147783 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325d899-cd80-46e1-9892-976fc02a2be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:29.148248 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325d899-cd80-46e1-9892-976fc02a2be7 None admin] PATH_INFO: `/v3/users/57f1e4c0e3bb46ee9871ed96d96328fa` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:29.150092 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e325d899-cd80-46e1-9892-976fc02a2be7 None admin] RBAC: Authorizing `identity:get_user(user_id=57f1e4c0e3bb46ee9871ed96d96328fa)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:29.151826 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e325d899-cd80-46e1-9892-976fc02a2be7 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:29.154176 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 95/204] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:28 2022] GET /identity/v3/users/57f1e4c0e3bb46ee9871ed96d96328fa => generated 289 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:46:29.154315 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0164a3-877d-4252-913a-2bbf3a5ceb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:29.154801 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0164a3-877d-4252-913a-2bbf3a5ceb41 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:29.155083 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0164a3-877d-4252-913a-2bbf3a5ceb41 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:29.155355 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0164a3-877d-4252-913a-2bbf3a5ceb41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:29.156122 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0164a3-877d-4252-913a-2bbf3a5ceb41 None admin] RBAC: Adding query filter params (role.id=13072ad4b81649459f59b8137db492ec, scope.project.id=e13bca418ede487ab7d1230e28403173, user.id=fcafeca38d4942cf9f052bd972453739) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:29.156439 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0164a3-877d-4252-913a-2bbf3a5ceb41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:29.157891 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0164a3-877d-4252-913a-2bbf3a5ceb41 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:29.159812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325d899-cd80-46e1-9892-976fc02a2be7 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:29.165811 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 110/205] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:46:28 2022] GET /identity/v3/role_assignments?user.id=fcafeca38d4942cf9f052bd972453739&scope.project.id=e13bca418ede487ab7d1230e28403173&role.id=13072ad4b81649459f59b8137db492ec => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:46:29.318797 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325d899-cd80-46e1-9892-976fc02a2be7 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:29.354084 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:29.368992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f2b0e-98f2-4dc4-866f-252bc4fb7adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:29.370184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f2b0e-98f2-4dc4-866f-252bc4fb7adc None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:29.370184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f2b0e-98f2-4dc4-866f-252bc4fb7adc None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:29.370184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f2b0e-98f2-4dc4-866f-252bc4fb7adc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:29.455882 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61f2b0e-98f2-4dc4-866f-252bc4fb7adc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:29.457805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61f2b0e-98f2-4dc4-866f-252bc4fb7adc None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:29.564538 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 96/206] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:46:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:46:29.570612 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0164a3-877d-4252-913a-2bbf3a5ceb41 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:29.689484 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:29.727252 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0164a3-877d-4252-913a-2bbf3a5ceb41 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:29.776120 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb624889-9ce2-463e-a871-791169e61421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:29.777000 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb624889-9ce2-463e-a871-791169e61421 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:29.777459 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb624889-9ce2-463e-a871-791169e61421 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:29.777880 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb624889-9ce2-463e-a871-791169e61421 None admin] PATH_INFO: `/v3/projects/1464b3dd76734fb593f3ed3582957e86` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:29.779182 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb624889-9ce2-463e-a871-791169e61421 None admin] RBAC: Authorizing `identity:get_project(project_id=1464b3dd76734fb593f3ed3582957e86)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:29.780793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb624889-9ce2-463e-a871-791169e61421 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:29.782576 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 111/207] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:29 2022] GET /identity/v3/projects/1464b3dd76734fb593f3ed3582957e86 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:29.788612 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f2b0e-98f2-4dc4-866f-252bc4fb7adc None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:29.826058 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:29.922852 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f2b0e-98f2-4dc4-866f-252bc4fb7adc None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:29.961664 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0c89c-de73-4a0e-88fb-3f97441b988b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:29.962287 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0c89c-de73-4a0e-88fb-3f97441b988b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:29.962565 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0c89c-de73-4a0e-88fb-3f97441b988b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:29.962840 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0c89c-de73-4a0e-88fb-3f97441b988b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:29.963580 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d0c89c-de73-4a0e-88fb-3f97441b988b None admin] RBAC: Adding query filter params (role.id=e01870d6deef4f1ebbc7470c342046a6, scope.project.id=1464b3dd76734fb593f3ed3582957e86, user.id=57f1e4c0e3bb46ee9871ed96d96328fa) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:29.963892 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d0c89c-de73-4a0e-88fb-3f97441b988b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:29.965515 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d0c89c-de73-4a0e-88fb-3f97441b988b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:29.971918 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 97/208] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:46:29 2022] GET /identity/v3/role_assignments?user.id=57f1e4c0e3bb46ee9871ed96d96328fa&scope.project.id=1464b3dd76734fb593f3ed3582957e86&role.id=e01870d6deef4f1ebbc7470c342046a6 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:31.270041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34476504-77e5-4d38-9e03-83c88c5adb1c None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:31.270559 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8f9eb-37fb-4d40-a575-16971d0def22 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:31.270895 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8f9eb-37fb-4d40-a575-16971d0def22 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:31.270895 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8f9eb-37fb-4d40-a575-16971d0def22 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:31.271584 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34476504-77e5-4d38-9e03-83c88c5adb1c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:31.272120 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34476504-77e5-4d38-9e03-83c88c5adb1c None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:31.272662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 98/209] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:31.273945 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 112/210] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:31.279573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aff92c-9ec4-441c-91a9-02e0e4b24cc7 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:31.279848 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aff92c-9ec4-441c-91a9-02e0e4b24cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:31.280153 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aff92c-9ec4-441c-91a9-02e0e4b24cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:31.280743 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e77480-5df9-470a-9a5e-2825b561ac33 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:31.281241 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e77480-5df9-470a-9a5e-2825b561ac33 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:31.281708 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e77480-5df9-470a-9a5e-2825b561ac33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:31.301139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-11aff92c-9ec4-441c-91a9-02e0e4b24cc7 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:31.302692 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-11aff92c-9ec4-441c-91a9-02e0e4b24cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:31.304297 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-e2e77480-5df9-470a-9a5e-2825b561ac33 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:31.305917 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e2e77480-5df9-470a-9a5e-2825b561ac33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:31.381897 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 99/211] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:31.384349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602c383-7429-496c-b960-f4f94f33cb86 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:31.384349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602c383-7429-496c-b960-f4f94f33cb86 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:31.384349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602c383-7429-496c-b960-f4f94f33cb86 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:31.385612 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 100/212] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:31.387619 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 113/213] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:46:31.390600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f846adf-a329-46fe-8d55-41cb5cd252bb None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:31.390881 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f846adf-a329-46fe-8d55-41cb5cd252bb None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:31.391156 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f846adf-a329-46fe-8d55-41cb5cd252bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:31.409139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-7f846adf-a329-46fe-8d55-41cb5cd252bb None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:31.410323 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7f846adf-a329-46fe-8d55-41cb5cd252bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:31.419888 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:31.481360 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 101/214] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:31.650569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e77480-5df9-470a-9a5e-2825b561ac33 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:31.666108 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f846adf-a329-46fe-8d55-41cb5cd252bb None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:31.726397 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:31.757880 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e77480-5df9-470a-9a5e-2825b561ac33 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:31.776814 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f846adf-a329-46fe-8d55-41cb5cd252bb None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:31.791796 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076acf92-9e57-46e9-8b13-69daf3d5ff47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:31.792466 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076acf92-9e57-46e9-8b13-69daf3d5ff47 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:31.792751 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076acf92-9e57-46e9-8b13-69daf3d5ff47 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:31.793029 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076acf92-9e57-46e9-8b13-69daf3d5ff47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:31.802755 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076acf92-9e57-46e9-8b13-69daf3d5ff47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:31.808665 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-076acf92-9e57-46e9-8b13-69daf3d5ff47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:46:31.808665 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:31.808665 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:31.808665 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:31.808665 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:31.808665 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:31.808665 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:31.808665 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:31.808665 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:31.808665 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:31.808665 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:31.808665 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:31.808665 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:31.808665 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:31.808665 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:31.808665 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:31.809794 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:31.809794 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:31.809794 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:31.809794 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:31.809794 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:31.809794 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:31.809794 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:31.809794 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:31.809794 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:31.809794 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:31.809794 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:31.809794 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:31.809794 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:31.809794 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:31.809794 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:31.809794 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:31.810772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:31.810772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:31.810772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:31.810772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:31.810772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:31.810772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:31.810772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:31.810772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:31.810772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:31.810772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:31.810772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:46:31.810772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:46:31.810772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 114/215] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:46:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:31.811605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6b238-7251-47e0-ae34-63ba2837597d None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:31.811605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6b238-7251-47e0-ae34-63ba2837597d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:31.811786 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6b238-7251-47e0-ae34-63ba2837597d None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:31.812524 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a36b86-2f60-4bce-b434-bd1a2d5953fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:31.813164 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 115/216] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:31.813271 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a36b86-2f60-4bce-b434-bd1a2d5953fa None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:31.813492 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a36b86-2f60-4bce-b434-bd1a2d5953fa None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:31.813771 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a36b86-2f60-4bce-b434-bd1a2d5953fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:31.816924 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6b238-7251-47e0-ae34-63ba2837597d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:31.825620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a36b86-2f60-4bce-b434-bd1a2d5953fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:31.830523 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-d3a36b86-2f60-4bce-b434-bd1a2d5953fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:31.830523 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:31.830523 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:31.830523 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:31.830523 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:31.830523 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:31.830523 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:31.830523 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:31.830523 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:31.830523 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:31.830523 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:31.830523 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:31.830523 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:31.830523 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:31.830523 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:31.830523 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:31.831421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:31.831421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:31.831421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:31.831421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:31.831421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:31.831421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:31.831421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:31.831421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:31.831421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:31.831421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:31.831421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:31.831421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:31.831421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:31.831421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:31.831421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:31.831421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:31.832312 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:31.832312 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:31.832312 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:31.832312 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:31.832312 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:31.832312 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:31.832312 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:31.832312 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:31.832312 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:31.832312 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:31.832312 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:31.832312 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:46:31.832312 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 102/217] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:46:31.832974 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a36b86-2f60-4bce-b434-bd1a2d5953fa None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:31.838003 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:31.918747 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6b238-7251-47e0-ae34-63ba2837597d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:31.936088 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a36b86-2f60-4bce-b434-bd1a2d5953fa None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:31.946773 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:31.952324 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45434874-0394-4f9a-ad89-5439ea745cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:31.952955 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45434874-0394-4f9a-ad89-5439ea745cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:31.958186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45434874-0394-4f9a-ad89-5439ea745cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:31.958186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45434874-0394-4f9a-ad89-5439ea745cc3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:31.958186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45434874-0394-4f9a-ad89-5439ea745cc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:31.958186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45434874-0394-4f9a-ad89-5439ea745cc3 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:31.958186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 116/218] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:46:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:31.958692 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0af3e-5514-4ad9-9cc1-f6b40fdb6eca None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:31.959125 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0af3e-5514-4ad9-9cc1-f6b40fdb6eca None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:31.959395 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0af3e-5514-4ad9-9cc1-f6b40fdb6eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:31.973020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26d009-27ae-42bb-b814-51aed8c34faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:31.973631 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26d009-27ae-42bb-b814-51aed8c34faa None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:31.973912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26d009-27ae-42bb-b814-51aed8c34faa None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:31.974184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26d009-27ae-42bb-b814-51aed8c34faa None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:31.974858 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d26d009-27ae-42bb-b814-51aed8c34faa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:31.975161 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d26d009-27ae-42bb-b814-51aed8c34faa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:31.976348 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-03a0af3e-5514-4ad9-9cc1-f6b40fdb6eca None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:31.976647 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d26d009-27ae-42bb-b814-51aed8c34faa None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:31.978914 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-03a0af3e-5514-4ad9-9cc1-f6b40fdb6eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:31.988171 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 103/219] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:31.993807 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26d009-27ae-42bb-b814-51aed8c34faa None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:32.045009 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 117/220] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:32.046121 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a0af3e-5514-4ad9-9cc1-f6b40fdb6eca None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:32.060584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:32.097841 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26d009-27ae-42bb-b814-51aed8c34faa None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:32.133622 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a814653-5d66-43ef-b968-35782cdd0f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:32.134445 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a814653-5d66-43ef-b968-35782cdd0f76 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:32.134727 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a814653-5d66-43ef-b968-35782cdd0f76 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:32.134998 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a814653-5d66-43ef-b968-35782cdd0f76 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:32.135662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a814653-5d66-43ef-b968-35782cdd0f76 None admin] RBAC: Adding query filter params (name=member) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:32.135966 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a814653-5d66-43ef-b968-35782cdd0f76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:32.137391 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a814653-5d66-43ef-b968-35782cdd0f76 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:32.146151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 104/221] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Sep 9 20:46:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:32.147153 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a814653-5d66-43ef-b968-35782cdd0f76 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:32.153458 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a0af3e-5514-4ad9-9cc1-f6b40fdb6eca None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:32.181952 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:32.189390 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a220a0c-faa1-4825-b044-83995bd493e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:32.190009 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a220a0c-faa1-4825-b044-83995bd493e0 None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:32.190283 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a220a0c-faa1-4825-b044-83995bd493e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:32.200164 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a220a0c-faa1-4825-b044-83995bd493e0 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:32.200164 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a220a0c-faa1-4825-b044-83995bd493e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:32.200164 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a220a0c-faa1-4825-b044-83995bd493e0 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:32.250079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a814653-5d66-43ef-b968-35782cdd0f76 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:32.275198 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:32.282585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb7225e-a2f3-4159-8ff8-dd9cca025e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:32.283209 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7225e-a2f3-4159-8ff8-dd9cca025e7b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:32.283488 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7225e-a2f3-4159-8ff8-dd9cca025e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:32.283758 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7225e-a2f3-4159-8ff8-dd9cca025e7b None admin] PATH_INFO: `/v3/users/fcafeca38d4942cf9f052bd972453739` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:32.285043 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb7225e-a2f3-4159-8ff8-dd9cca025e7b None admin] RBAC: Authorizing `identity:get_user(user_id=fcafeca38d4942cf9f052bd972453739)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:32.287087 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb7225e-a2f3-4159-8ff8-dd9cca025e7b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:32.293951 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 105/222] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:32 2022] GET /identity/v3/users/fcafeca38d4942cf9f052bd972453739 => generated 283 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:32.293951 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb7225e-a2f3-4159-8ff8-dd9cca025e7b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:32.298140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 118/223] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Sep 9 20:46:32 2022] POST /identity/v3/users => generated 297 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:46:32.298140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a220a0c-faa1-4825-b044-83995bd493e0 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:32.413779 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a220a0c-faa1-4825-b044-83995bd493e0 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:32.417932 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb7225e-a2f3-4159-8ff8-dd9cca025e7b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:32.453699 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff065eda-56f1-4af6-83a9-e66e2d8832de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:32.454458 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff065eda-56f1-4af6-83a9-e66e2d8832de None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:32.454839 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff065eda-56f1-4af6-83a9-e66e2d8832de None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:32.455222 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff065eda-56f1-4af6-83a9-e66e2d8832de None admin] PATH_INFO: `/v3/projects/e13bca418ede487ab7d1230e28403173` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:32.456187 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d7b2c-d839-4e03-b053-0485fd3fe385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:32.456492 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff065eda-56f1-4af6-83a9-e66e2d8832de None admin] RBAC: Authorizing `identity:get_project(project_id=e13bca418ede487ab7d1230e28403173)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:32.456826 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d7b2c-d839-4e03-b053-0485fd3fe385 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:32.457104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d7b2c-d839-4e03-b053-0485fd3fe385 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:32.457374 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d7b2c-d839-4e03-b053-0485fd3fe385 None admin] PATH_INFO: `/v3/users/37f9da3b9a924300acf8f151691f0feb` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:32.458115 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff065eda-56f1-4af6-83a9-e66e2d8832de None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:32.459897 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 119/224] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:32 2022] GET /identity/v3/projects/e13bca418ede487ab7d1230e28403173 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:32.465492 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff065eda-56f1-4af6-83a9-e66e2d8832de None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:32.474533 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41d7b2c-d839-4e03-b053-0485fd3fe385 None admin] RBAC: Authorizing `identity:get_user(user_id=37f9da3b9a924300acf8f151691f0feb)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:32.475308 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41d7b2c-d839-4e03-b053-0485fd3fe385 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:32.477021 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 106/225] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:32 2022] GET /identity/v3/users/37f9da3b9a924300acf8f151691f0feb => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:32.478292 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d7b2c-d839-4e03-b053-0485fd3fe385 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:32.532205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:32.543318 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:32.588503 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff065eda-56f1-4af6-83a9-e66e2d8832de None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:32.599606 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d7b2c-d839-4e03-b053-0485fd3fe385 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:32.637537 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665766f1-b066-4812-aeda-df163756867a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:32.638189 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665766f1-b066-4812-aeda-df163756867a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:32.638469 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665766f1-b066-4812-aeda-df163756867a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:32.638741 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665766f1-b066-4812-aeda-df163756867a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:32.642432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f33b6b-f663-4a7e-9dda-6621af4b5a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:32.643102 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f33b6b-f663-4a7e-9dda-6621af4b5a15 None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:32.643379 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f33b6b-f663-4a7e-9dda-6621af4b5a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:32.643649 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f33b6b-f663-4a7e-9dda-6621af4b5a15 None admin] PATH_INFO: `/v3/projects/e13bca418ede487ab7d1230e28403173/users/fcafeca38d4942cf9f052bd972453739/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:32.645816 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f33b6b-f663-4a7e-9dda-6621af4b5a15 None admin] RBAC: Authorizing `identity:create_grant(project_id=e13bca418ede487ab7d1230e28403173, user_id=fcafeca38d4942cf9f052bd972453739, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:32.647337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f33b6b-f663-4a7e-9dda-6621af4b5a15 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:32.652368 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665766f1-b066-4812-aeda-df163756867a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:32.657787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-665766f1-b066-4812-aeda-df163756867a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:46:32.657787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:32.657787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:32.657787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:32.657787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:32.657787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:32.657787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:32.657787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:32.657787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:32.657787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:32.657787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:32.657787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:32.657787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:32.657787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:32.657787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:32.657787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:32.658675 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:32.658675 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:32.658675 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:32.658675 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:32.658675 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:32.658675 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:32.658675 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:32.658675 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:32.658675 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:32.658675 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:32.658675 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:32.658675 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:32.658675 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:32.658675 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:32.658675 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:32.658675 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:32.659731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:32.659731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:32.659731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:32.659731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:32.659731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:32.659731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:32.659731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:32.659731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:32.659731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:32.659731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:32.659731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:46:32.659731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:46:32.659731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 120/226] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Sep 9 20:46:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:32.659731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665766f1-b066-4812-aeda-df163756867a None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:32.724158 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 107/227] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Sep 9 20:46:32 2022] PUT /identity/v3/projects/e13bca418ede487ab7d1230e28403173/users/fcafeca38d4942cf9f052bd972453739/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:46:32.728187 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f33b6b-f663-4a7e-9dda-6621af4b5a15 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:32.806328 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665766f1-b066-4812-aeda-df163756867a None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:32.830810 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:32.892823 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b67948-ab3c-4a3d-9f20-5dfec0bb0fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:32.893458 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b67948-ab3c-4a3d-9f20-5dfec0bb0fd1 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:32.893739 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b67948-ab3c-4a3d-9f20-5dfec0bb0fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:32.894015 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b67948-ab3c-4a3d-9f20-5dfec0bb0fd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:32.913930 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f33b6b-f663-4a7e-9dda-6621af4b5a15 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:32.934987 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:32.959663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e37f91c-4c28-45f1-8737-238c6c2eb5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:32.960325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37f91c-4c28-45f1-8737-238c6c2eb5a1 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:32.960606 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37f91c-4c28-45f1-8737-238c6c2eb5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:32.960882 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37f91c-4c28-45f1-8737-238c6c2eb5a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:32.962103 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e37f91c-4c28-45f1-8737-238c6c2eb5a1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:32.962413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e37f91c-4c28-45f1-8737-238c6c2eb5a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:32.964811 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e37f91c-4c28-45f1-8737-238c6c2eb5a1 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:32.975983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 108/228] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:46:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:32.982728 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e37f91c-4c28-45f1-8737-238c6c2eb5a1 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:33.045088 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b67948-ab3c-4a3d-9f20-5dfec0bb0fd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:33.046601 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b67948-ab3c-4a3d-9f20-5dfec0bb0fd1 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:33.157882 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 121/229] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:46:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:46:33.165924 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b67948-ab3c-4a3d-9f20-5dfec0bb0fd1 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:33.187542 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:33.262367 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e37f91c-4c28-45f1-8737-238c6c2eb5a1 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:33.281726 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b67948-ab3c-4a3d-9f20-5dfec0bb0fd1 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:33.322569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8493353-3347-4135-80e9-6abeb84f1452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:33.323204 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8493353-3347-4135-80e9-6abeb84f1452 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:33.323481 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8493353-3347-4135-80e9-6abeb84f1452 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:33.323806 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8493353-3347-4135-80e9-6abeb84f1452 None admin] PATH_INFO: `/v3/projects/e13bca418ede487ab7d1230e28403173` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:33.325471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8493353-3347-4135-80e9-6abeb84f1452 None admin] RBAC: Authorizing `identity:get_project(project_id=e13bca418ede487ab7d1230e28403173)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:33.327306 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8493353-3347-4135-80e9-6abeb84f1452 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:33.329036 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 122/230] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:33 2022] GET /identity/v3/projects/e13bca418ede487ab7d1230e28403173 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:33.333343 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872dd0f-1405-4575-8231-807e001856f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:33.334056 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872dd0f-1405-4575-8231-807e001856f5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:33.334343 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872dd0f-1405-4575-8231-807e001856f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:33.334617 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872dd0f-1405-4575-8231-807e001856f5 None admin] PATH_INFO: `/v3/users/fcafeca38d4942cf9f052bd972453739` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:33.335939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8493353-3347-4135-80e9-6abeb84f1452 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:33.337459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f872dd0f-1405-4575-8231-807e001856f5 None admin] RBAC: Authorizing `identity:get_user(user_id=fcafeca38d4942cf9f052bd972453739)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:33.338906 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f872dd0f-1405-4575-8231-807e001856f5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:33.349387 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 109/231] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:32 2022] GET /identity/v3/users/fcafeca38d4942cf9f052bd972453739 => generated 283 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:46:33.356596 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872dd0f-1405-4575-8231-807e001856f5 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:33.443489 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:33.516760 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:33.547202 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872dd0f-1405-4575-8231-807e001856f5 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:33.555257 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8493353-3347-4135-80e9-6abeb84f1452 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:33.593610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6552fe5-fc49-4f6b-adfb-a7532a89db4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:33.594414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6552fe5-fc49-4f6b-adfb-a7532a89db4c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:33.594761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6552fe5-fc49-4f6b-adfb-a7532a89db4c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:33.595042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6552fe5-fc49-4f6b-adfb-a7532a89db4c None admin] PATH_INFO: `/v3/projects/e13bca418ede487ab7d1230e28403173` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:33.596214 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6552fe5-fc49-4f6b-adfb-a7532a89db4c None admin] RBAC: Authorizing `identity:get_project(project_id=e13bca418ede487ab7d1230e28403173)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:33.598342 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6552fe5-fc49-4f6b-adfb-a7532a89db4c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:33.600045 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 110/232] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:33 2022] GET /identity/v3/projects/e13bca418ede487ab7d1230e28403173 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:33.606346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6552fe5-fc49-4f6b-adfb-a7532a89db4c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:33.645562 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15e83c-6258-4cb1-9882-51a2e67d9165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:33.646185 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15e83c-6258-4cb1-9882-51a2e67d9165 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:33.646460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15e83c-6258-4cb1-9882-51a2e67d9165 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:33.646730 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15e83c-6258-4cb1-9882-51a2e67d9165 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:33.647468 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca15e83c-6258-4cb1-9882-51a2e67d9165 None admin] RBAC: Adding query filter params (role.id=c59d891d042e48bab625f7b3666ef536, scope.project.id=e13bca418ede487ab7d1230e28403173, user.id=37f9da3b9a924300acf8f151691f0feb) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:33.647778 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca15e83c-6258-4cb1-9882-51a2e67d9165 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:33.649406 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca15e83c-6258-4cb1-9882-51a2e67d9165 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:33.660888 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 123/233] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:46:33 2022] GET /identity/v3/role_assignments?user.id=37f9da3b9a924300acf8f151691f0feb&scope.project.id=e13bca418ede487ab7d1230e28403173&role.id=c59d891d042e48bab625f7b3666ef536 => generated 259 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:33.749685 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6552fe5-fc49-4f6b-adfb-a7532a89db4c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:33.771177 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:33.795549 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2839c18e-8519-468c-929f-1b95b92a2288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:33.796249 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839c18e-8519-468c-929f-1b95b92a2288 None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:33.796536 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839c18e-8519-468c-929f-1b95b92a2288 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:33.796814 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839c18e-8519-468c-929f-1b95b92a2288 None admin] PATH_INFO: `/v3/projects/e13bca418ede487ab7d1230e28403173/users/fcafeca38d4942cf9f052bd972453739/roles/13072ad4b81649459f59b8137db492ec` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:33.805999 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2839c18e-8519-468c-929f-1b95b92a2288 None admin] RBAC: Authorizing `identity:create_grant(project_id=e13bca418ede487ab7d1230e28403173, user_id=fcafeca38d4942cf9f052bd972453739, role_id=13072ad4b81649459f59b8137db492ec)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:33.807586 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2839c18e-8519-468c-929f-1b95b92a2288 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:33.913550 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 111/234] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Sep 9 20:46:33 2022] PUT /identity/v3/projects/e13bca418ede487ab7d1230e28403173/users/fcafeca38d4942cf9f052bd972453739/roles/13072ad4b81649459f59b8137db492ec => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:46:34.898089 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa7c12-f82e-48d4-9398-f7fdc948c50b None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:34.898613 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa7c12-f82e-48d4-9398-f7fdc948c50b None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:34.898613 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa7c12-f82e-48d4-9398-f7fdc948c50b None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:34.900617 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 124/235] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:34.905862 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897ca34-7cfb-4903-b2f3-c89dad9ba159 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:34.906278 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897ca34-7cfb-4903-b2f3-c89dad9ba159 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:34.906464 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897ca34-7cfb-4903-b2f3-c89dad9ba159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:34.918959 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffddc5ae-bf8f-4d5c-b008-9b3b7c5140e4 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:34.919240 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffddc5ae-bf8f-4d5c-b008-9b3b7c5140e4 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:34.919503 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffddc5ae-bf8f-4d5c-b008-9b3b7c5140e4 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:34.920860 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 125/236] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:34.925828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15c3cd-3e72-4492-b4c8-e5972b912c7a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:34.926131 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15c3cd-3e72-4492-b4c8-e5972b912c7a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:34.926404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15c3cd-3e72-4492-b4c8-e5972b912c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:34.945196 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e897ca34-7cfb-4903-b2f3-c89dad9ba159 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:34.946434 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-6b15c3cd-3e72-4492-b4c8-e5972b912c7a None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:34.946952 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e897ca34-7cfb-4903-b2f3-c89dad9ba159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:34.947602 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-6b15c3cd-3e72-4492-b4c8-e5972b912c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:35.022743 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 126/237] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:46:35.074176 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 112/238] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:35.192150 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6af841-4d4d-4034-9065-dda3d74fd918 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:35.192442 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6af841-4d4d-4034-9065-dda3d74fd918 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:35.192663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6af841-4d4d-4034-9065-dda3d74fd918 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:35.194020 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 127/239] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:35.199904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea10e85-5634-4b57-b2c4-df22ecfab3a7 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:35.200501 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea10e85-5634-4b57-b2c4-df22ecfab3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:35.200990 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea10e85-5634-4b57-b2c4-df22ecfab3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:35.226330 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-4ea10e85-5634-4b57-b2c4-df22ecfab3a7 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:35.227881 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4ea10e85-5634-4b57-b2c4-df22ecfab3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:35.307775 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 113/240] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:46:35.412794 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6af841-4d4d-4034-9065-dda3d74fd918 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:35.468129 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea10e85-5634-4b57-b2c4-df22ecfab3a7 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:35.494782 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:35.508588 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:35.532225 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6af841-4d4d-4034-9065-dda3d74fd918 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:35.604774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbdfb0a-c687-42d7-a503-b82dc60de346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:35.605421 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdfb0a-c687-42d7-a503-b82dc60de346 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:35.605698 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdfb0a-c687-42d7-a503-b82dc60de346 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:35.605972 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdfb0a-c687-42d7-a503-b82dc60de346 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:35.617416 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbdfb0a-c687-42d7-a503-b82dc60de346 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:35.623231 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-6dbdfb0a-c687-42d7-a503-b82dc60de346 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:46:35.623231 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:35.623231 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:35.623231 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:35.623231 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:35.623231 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:35.623231 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:35.623231 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:35.623231 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:35.623231 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:35.623231 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:35.623231 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:35.623231 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:35.623231 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:35.623231 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:35.623231 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:35.624130 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:35.624130 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:35.624130 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:35.624130 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:35.624130 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:35.624130 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:35.624130 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:35.624130 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:35.624130 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:35.624130 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:35.624130 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:35.624130 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:35.624130 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:35.624130 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:35.624130 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:35.624130 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:35.624933 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:35.624933 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:35.624933 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:35.624933 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:35.624933 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:35.624933 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:35.624933 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:35.624933 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:35.624933 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:35.624933 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:35.624933 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:46:35.624933 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:46:35.624933 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 128/241] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:35.630368 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbdfb0a-c687-42d7-a503-b82dc60de346 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:35.639912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea10e85-5634-4b57-b2c4-df22ecfab3a7 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:35.680902 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e06440-3174-4431-91c1-89a2cadaf6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:35.681547 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e06440-3174-4431-91c1-89a2cadaf6ff None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:35.681823 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e06440-3174-4431-91c1-89a2cadaf6ff None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:35.682093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e06440-3174-4431-91c1-89a2cadaf6ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:35.683217 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e06440-3174-4431-91c1-89a2cadaf6ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:35.684961 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e06440-3174-4431-91c1-89a2cadaf6ff None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:35.691202 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 114/242] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:46:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:46:35.692059 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e06440-3174-4431-91c1-89a2cadaf6ff None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:35.744562 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbdfb0a-c687-42d7-a503-b82dc60de346 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:35.792868 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5793900-c8aa-4964-98de-3cc9dab875da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:35.793482 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5793900-c8aa-4964-98de-3cc9dab875da None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:35.793762 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5793900-c8aa-4964-98de-3cc9dab875da None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:35.794035 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5793900-c8aa-4964-98de-3cc9dab875da None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:35.794709 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5793900-c8aa-4964-98de-3cc9dab875da None admin] RBAC: Adding query filter params (name=reader) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:35.795013 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5793900-c8aa-4964-98de-3cc9dab875da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:35.796418 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5793900-c8aa-4964-98de-3cc9dab875da None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:35.798074 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:35.801305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:35.806256 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e06440-3174-4431-91c1-89a2cadaf6ff None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:35.807105 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 129/243] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Sep 9 20:46:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:35.807776 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5793900-c8aa-4964-98de-3cc9dab875da None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:35.842059 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5267aa-377a-4265-8269-b36861b8068b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:35.842699 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5267aa-377a-4265-8269-b36861b8068b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:35.842975 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5267aa-377a-4265-8269-b36861b8068b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:35.843251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5267aa-377a-4265-8269-b36861b8068b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:35.855442 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5267aa-377a-4265-8269-b36861b8068b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:35.860358 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-4e5267aa-377a-4265-8269-b36861b8068b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:46:35.860358 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:35.860358 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:35.860358 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:35.860358 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:35.860358 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:35.860358 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:35.860358 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:35.860358 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:35.860358 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:35.860358 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:35.860358 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:35.860358 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:35.860358 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:35.860358 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:35.860358 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:35.861470 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:35.861470 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:35.861470 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:35.861470 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:35.861470 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:35.861470 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:35.861470 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:35.861470 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:35.861470 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:35.861470 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:35.861470 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:35.861470 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:35.861470 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:35.861470 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:35.861470 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:35.861470 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:35.862267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:35.862267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:35.862267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:35.862267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:35.862267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:35.862267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:35.862267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:35.862267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:35.862267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:35.862267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:35.862267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:46:35.862267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:46:35.862267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 115/244] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:46:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:35.862928 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5267aa-377a-4265-8269-b36861b8068b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:35.952579 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5793900-c8aa-4964-98de-3cc9dab875da None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:35.982313 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5267aa-377a-4265-8269-b36861b8068b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:36.004163 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0639b2d9-9e19-4fa9-82ec-5c0f2f7bffe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:36.005328 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639b2d9-9e19-4fa9-82ec-5c0f2f7bffe7 None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:36.006182 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639b2d9-9e19-4fa9-82ec-5c0f2f7bffe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:36.006928 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639b2d9-9e19-4fa9-82ec-5c0f2f7bffe7 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:36.007790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0639b2d9-9e19-4fa9-82ec-5c0f2f7bffe7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:36.009700 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0639b2d9-9e19-4fa9-82ec-5c0f2f7bffe7 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:36.018734 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc2b8b-4e46-4d37-ba28-c925c02cab74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:36.019366 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc2b8b-4e46-4d37-ba28-c925c02cab74 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:36.019642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc2b8b-4e46-4d37-ba28-c925c02cab74 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:36.019911 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc2b8b-4e46-4d37-ba28-c925c02cab74 None admin] PATH_INFO: `/v3/users/ccf3f53545074574bc218dad082d70cc` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:36.034312 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc2b8b-4e46-4d37-ba28-c925c02cab74 None admin] RBAC: Authorizing `identity:get_user(user_id=ccf3f53545074574bc218dad082d70cc)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:36.035818 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc2b8b-4e46-4d37-ba28-c925c02cab74 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:36.037590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 116/245] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:35 2022] GET /identity/v3/users/ccf3f53545074574bc218dad082d70cc => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:36.043660 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc2b8b-4e46-4d37-ba28-c925c02cab74 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:36.057759 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:36.099216 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 130/246] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Sep 9 20:46:35 2022] POST /identity/v3/users => generated 293 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:46:36.100716 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0639b2d9-9e19-4fa9-82ec-5c0f2f7bffe7 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:36.160853 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc2b8b-4e46-4d37-ba28-c925c02cab74 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:36.234341 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:36.239944 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc6d8c-d29d-4e99-9ae9-3a28a6da9815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:36.240681 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc6d8c-d29d-4e99-9ae9-3a28a6da9815 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:36.241800 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc6d8c-d29d-4e99-9ae9-3a28a6da9815 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:36.242097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc6d8c-d29d-4e99-9ae9-3a28a6da9815 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:36.253853 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc6d8c-d29d-4e99-9ae9-3a28a6da9815 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:36.253853 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc6d8c-d29d-4e99-9ae9-3a28a6da9815 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:36.253853 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc6d8c-d29d-4e99-9ae9-3a28a6da9815 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:36.265196 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 117/247] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:36.270573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc6d8c-d29d-4e99-9ae9-3a28a6da9815 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:36.322822 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0639b2d9-9e19-4fa9-82ec-5c0f2f7bffe7 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:36.366544 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8387554-899b-40f9-aaa2-785b7afe459e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:36.367308 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8387554-899b-40f9-aaa2-785b7afe459e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:36.367942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8387554-899b-40f9-aaa2-785b7afe459e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:36.368245 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8387554-899b-40f9-aaa2-785b7afe459e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:36.393298 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:36.415163 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc6d8c-d29d-4e99-9ae9-3a28a6da9815 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:36.447975 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8387554-899b-40f9-aaa2-785b7afe459e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:36.450377 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8387554-899b-40f9-aaa2-785b7afe459e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:36.455231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cde036-52dc-4f7c-8c0f-1decbfe0051a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:36.455876 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cde036-52dc-4f7c-8c0f-1decbfe0051a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:36.456929 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cde036-52dc-4f7c-8c0f-1decbfe0051a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:36.456929 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cde036-52dc-4f7c-8c0f-1decbfe0051a None admin] PATH_INFO: `/v3/users/fcafeca38d4942cf9f052bd972453739` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:36.457848 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cde036-52dc-4f7c-8c0f-1decbfe0051a None admin] RBAC: Authorizing `identity:get_user(user_id=fcafeca38d4942cf9f052bd972453739)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:36.459886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cde036-52dc-4f7c-8c0f-1decbfe0051a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:36.463507 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 118/248] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:36 2022] GET /identity/v3/users/fcafeca38d4942cf9f052bd972453739 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:46:36.465026 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bbf09-0394-48cc-bdc0-2f0cae2742d2 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:36.465559 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bbf09-0394-48cc-bdc0-2f0cae2742d2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:36.465850 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bbf09-0394-48cc-bdc0-2f0cae2742d2 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:36.468034 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 119/249] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:36.469787 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bbf09-0394-48cc-bdc0-2f0cae2742d2 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:36.521928 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:36.586464 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 131/250] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:46:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:46:36.588726 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3e456-c1d7-4e85-bc32-f8feb6a942cd None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:36.589245 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3e456-c1d7-4e85-bc32-f8feb6a942cd None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:36.589733 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3e456-c1d7-4e85-bc32-f8feb6a942cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:36.598393 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bbf09-0394-48cc-bdc0-2f0cae2742d2 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:36.610364 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-15b3e456-c1d7-4e85-bc32-f8feb6a942cd None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:36.611646 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-15b3e456-c1d7-4e85-bc32-f8feb6a942cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:36.636234 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4be5c9-e1c6-4502-9990-cee8049bfbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:36.637039 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4be5c9-e1c6-4502-9990-cee8049bfbf9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:36.637156 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4be5c9-e1c6-4502-9990-cee8049bfbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:36.637522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4be5c9-e1c6-4502-9990-cee8049bfbf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:36.662525 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:36.691359 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 132/251] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:36.692395 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3e456-c1d7-4e85-bc32-f8feb6a942cd None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:36.757478 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4be5c9-e1c6-4502-9990-cee8049bfbf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:36.760367 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4be5c9-e1c6-4502-9990-cee8049bfbf9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:36.859230 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:36.900252 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3e456-c1d7-4e85-bc32-f8feb6a942cd None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:36.941491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2a929-d818-4ebc-8fc8-f30439dda1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:36.942298 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2a929-d818-4ebc-8fc8-f30439dda1fb None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:36.942694 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2a929-d818-4ebc-8fc8-f30439dda1fb None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:36.943062 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2a929-d818-4ebc-8fc8-f30439dda1fb None admin] PATH_INFO: `/v3/projects/e13bca418ede487ab7d1230e28403173` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:36.944346 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e2a929-d818-4ebc-8fc8-f30439dda1fb None admin] RBAC: Authorizing `identity:get_project(project_id=e13bca418ede487ab7d1230e28403173)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:36.946537 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e2a929-d818-4ebc-8fc8-f30439dda1fb None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:36.948371 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 133/252] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:36 2022] GET /identity/v3/projects/e13bca418ede487ab7d1230e28403173 => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:36.949755 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c97a7f-ec88-4a62-8228-3fc3e3356eaf None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:36.950137 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c97a7f-ec88-4a62-8228-3fc3e3356eaf None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:36.950508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c97a7f-ec88-4a62-8228-3fc3e3356eaf None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:36.951893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 134/253] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:36.957611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee3dbf-385f-4ff3-8579-3ddfc9504975 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:36.958427 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee3dbf-385f-4ff3-8579-3ddfc9504975 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:36.958427 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee3dbf-385f-4ff3-8579-3ddfc9504975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:36.962355 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 120/254] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:46:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:46:36.963169 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4be5c9-e1c6-4502-9990-cee8049bfbf9 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:36.997232 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-7fee3dbf-385f-4ff3-8579-3ddfc9504975 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:36.998846 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-7fee3dbf-385f-4ff3-8579-3ddfc9504975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:37.036166 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:37.121985 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 135/255] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:37.122764 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee3dbf-385f-4ff3-8579-3ddfc9504975 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:37.135512 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4be5c9-e1c6-4502-9990-cee8049bfbf9 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:37.179390 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7925280-e9a8-4fc8-96d7-0e7f6ee2a34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:37.179390 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7925280-e9a8-4fc8-96d7-0e7f6ee2a34f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:37.179390 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7925280-e9a8-4fc8-96d7-0e7f6ee2a34f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:37.179390 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7925280-e9a8-4fc8-96d7-0e7f6ee2a34f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:37.179785 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7925280-e9a8-4fc8-96d7-0e7f6ee2a34f None admin] RBAC: Adding query filter params (role.id=e01870d6deef4f1ebbc7470c342046a6, scope.project.id=e13bca418ede487ab7d1230e28403173, user.id=ccf3f53545074574bc218dad082d70cc) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:37.179785 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7925280-e9a8-4fc8-96d7-0e7f6ee2a34f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:37.181056 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7925280-e9a8-4fc8-96d7-0e7f6ee2a34f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:37.197943 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 121/256] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:46:36 2022] GET /identity/v3/role_assignments?user.id=ccf3f53545074574bc218dad082d70cc&scope.project.id=e13bca418ede487ab7d1230e28403173&role.id=e01870d6deef4f1ebbc7470c342046a6 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:37.198682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7925280-e9a8-4fc8-96d7-0e7f6ee2a34f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:37.240473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:37.307319 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee3dbf-385f-4ff3-8579-3ddfc9504975 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:37.376513 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7925280-e9a8-4fc8-96d7-0e7f6ee2a34f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:37.392370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925c97f-ace0-40d3-bef2-b6c89141230b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:37.393027 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925c97f-ace0-40d3-bef2-b6c89141230b None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:37.393306 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925c97f-ace0-40d3-bef2-b6c89141230b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:37.393577 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925c97f-ace0-40d3-bef2-b6c89141230b None admin] PATH_INFO: `/v3/projects/e13bca418ede487ab7d1230e28403173` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:37.394766 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925c97f-ace0-40d3-bef2-b6c89141230b None admin] RBAC: Authorizing `identity:get_project(project_id=e13bca418ede487ab7d1230e28403173)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:37.396280 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925c97f-ace0-40d3-bef2-b6c89141230b None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:37.397516 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:37.398165 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 136/257] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:37 2022] GET /identity/v3/projects/e13bca418ede487ab7d1230e28403173 => generated 302 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:37.404686 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925c97f-ace0-40d3-bef2-b6c89141230b None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:37.431515 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd283a35-b929-4e4b-974b-1205247cab81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:37.432186 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd283a35-b929-4e4b-974b-1205247cab81 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:37.432476 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd283a35-b929-4e4b-974b-1205247cab81 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:37.432749 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd283a35-b929-4e4b-974b-1205247cab81 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:37.458737 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd283a35-b929-4e4b-974b-1205247cab81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:37.470894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-fd283a35-b929-4e4b-974b-1205247cab81 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:37.470894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:37.470894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:37.470894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:37.470894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:37.470894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:37.470894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:37.470894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:37.470894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:37.470894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:37.470894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:37.470894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:37.470894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:37.470894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:37.470894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:37.470894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:37.471849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:37.471849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:37.471849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:37.471849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:37.471849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:37.471849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:37.471849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:37.471849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:37.471849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:37.471849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:37.471849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:37.471849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:37.471849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:37.471849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:37.471849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:37.471849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:37.472698 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:37.472698 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:37.472698 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:37.472698 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:37.472698 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:37.472698 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:37.472698 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:37.472698 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:37.472698 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:37.472698 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:37.472698 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:37.472698 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:46:37.472698 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 122/258] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:37.479529 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd283a35-b929-4e4b-974b-1205247cab81 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:37.572626 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925c97f-ace0-40d3-bef2-b6c89141230b None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:37.611771 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d390028-e061-4dc1-b166-b0da5955d5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:37.612414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d390028-e061-4dc1-b166-b0da5955d5ad None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:37.612695 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d390028-e061-4dc1-b166-b0da5955d5ad None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:37.612968 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d390028-e061-4dc1-b166-b0da5955d5ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:37.613707 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d390028-e061-4dc1-b166-b0da5955d5ad None admin] RBAC: Adding query filter params (role.id=fcbf519fbc84458f913671a5a1d0914c, scope.project.id=e13bca418ede487ab7d1230e28403173, user.id=fcafeca38d4942cf9f052bd972453739) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:37.614013 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d390028-e061-4dc1-b166-b0da5955d5ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:37.615450 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d390028-e061-4dc1-b166-b0da5955d5ad None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:37.635115 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 137/259] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:46:37 2022] GET /identity/v3/role_assignments?user.id=fcafeca38d4942cf9f052bd972453739&scope.project.id=e13bca418ede487ab7d1230e28403173&role.id=fcbf519fbc84458f913671a5a1d0914c => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:46:37.661719 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd283a35-b929-4e4b-974b-1205247cab81 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:37.678674 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d390028-e061-4dc1-b166-b0da5955d5ad None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:37.701380 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:37.712730 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289843e-e97e-4bb6-9156-d7fdc0e89365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:37.713360 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289843e-e97e-4bb6-9156-d7fdc0e89365 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:37.713646 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289843e-e97e-4bb6-9156-d7fdc0e89365 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:37.717013 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289843e-e97e-4bb6-9156-d7fdc0e89365 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:37.717013 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f289843e-e97e-4bb6-9156-d7fdc0e89365 None admin] RBAC: Adding query filter params (name=member) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:37.717013 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f289843e-e97e-4bb6-9156-d7fdc0e89365 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:37.717447 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f289843e-e97e-4bb6-9156-d7fdc0e89365 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:37.728821 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 123/260] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Sep 9 20:46:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:37.735884 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289843e-e97e-4bb6-9156-d7fdc0e89365 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:37.822520 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:37.879956 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d390028-e061-4dc1-b166-b0da5955d5ad None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:37.922674 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59efac6-bf5c-4e4d-b90f-0d0899ca0d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:37.922875 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59efac6-bf5c-4e4d-b90f-0d0899ca0d1b None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:37.923124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59efac6-bf5c-4e4d-b90f-0d0899ca0d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:37.923326 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59efac6-bf5c-4e4d-b90f-0d0899ca0d1b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:37.928685 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289843e-e97e-4bb6-9156-d7fdc0e89365 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:37.936791 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59efac6-bf5c-4e4d-b90f-0d0899ca0d1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:37.942430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-c59efac6-bf5c-4e4d-b90f-0d0899ca0d1b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:46:37.942430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:37.942430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:37.942430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:37.942430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:37.942430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:37.942430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:37.942430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:37.942430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:37.942430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:37.942430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:37.942430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:37.942430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:37.942430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:37.942430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:37.942430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:37.943375 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:37.943375 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:37.943375 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:37.943375 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:37.943375 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:37.943375 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:37.943375 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:37.943375 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:37.943375 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:37.943375 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:37.943375 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:37.943375 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:37.943375 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:37.943375 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:37.943375 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:37.943375 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:37.944199 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:37.944199 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:37.944199 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:37.944199 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:37.944199 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:37.944199 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:37.944199 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:37.944199 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:37.944199 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:37.944199 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:37.944199 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:46:37.944199 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:46:37.944199 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 138/261] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Sep 9 20:46:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:37.949532 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59efac6-bf5c-4e4d-b90f-0d0899ca0d1b None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:37.980661 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acff467-3720-4c55-b1ba-11570879f149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:37.981688 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acff467-3720-4c55-b1ba-11570879f149 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:37.982186 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acff467-3720-4c55-b1ba-11570879f149 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:37.982650 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acff467-3720-4c55-b1ba-11570879f149 None admin] PATH_INFO: `/v3/users/37f9da3b9a924300acf8f151691f0feb` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:37.984015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acff467-3720-4c55-b1ba-11570879f149 None admin] RBAC: Authorizing `identity:get_user(user_id=37f9da3b9a924300acf8f151691f0feb)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:37.985714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acff467-3720-4c55-b1ba-11570879f149 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:37.987568 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 124/262] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:37 2022] GET /identity/v3/users/37f9da3b9a924300acf8f151691f0feb => generated 297 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:46:38.006370 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acff467-3720-4c55-b1ba-11570879f149 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:38.038433 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:38.070432 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59efac6-bf5c-4e4d-b90f-0d0899ca0d1b None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:38.110989 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d889b07a-1415-4079-a9b8-87c0cf31deab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:38.111842 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889b07a-1415-4079-a9b8-87c0cf31deab None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:38.112358 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889b07a-1415-4079-a9b8-87c0cf31deab None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:38.112912 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889b07a-1415-4079-a9b8-87c0cf31deab None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:38.113878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d889b07a-1415-4079-a9b8-87c0cf31deab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:38.114310 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d889b07a-1415-4079-a9b8-87c0cf31deab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:38.115875 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d889b07a-1415-4079-a9b8-87c0cf31deab None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:38.126129 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 139/263] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:46:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:38.131851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d889b07a-1415-4079-a9b8-87c0cf31deab None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:38.190592 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acff467-3720-4c55-b1ba-11570879f149 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:38.192305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:38.238184 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d889b07a-1415-4079-a9b8-87c0cf31deab None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:38.252930 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aede50-cd53-450b-bce5-dbc4f85c7c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:38.253325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aede50-cd53-450b-bce5-dbc4f85c7c3e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:38.253610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aede50-cd53-450b-bce5-dbc4f85c7c3e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:38.253887 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aede50-cd53-450b-bce5-dbc4f85c7c3e None admin] PATH_INFO: `/v3/projects/e13bca418ede487ab7d1230e28403173` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:38.256598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aede50-cd53-450b-bce5-dbc4f85c7c3e None admin] RBAC: Authorizing `identity:get_project(project_id=e13bca418ede487ab7d1230e28403173)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:38.258242 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aede50-cd53-450b-bce5-dbc4f85c7c3e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:38.260521 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 125/264] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:38 2022] GET /identity/v3/projects/e13bca418ede487ab7d1230e28403173 => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:38.269765 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aede50-cd53-450b-bce5-dbc4f85c7c3e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:38.280557 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d9b68-3a5a-4edc-ad17-968f4ae756c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:38.281481 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d9b68-3a5a-4edc-ad17-968f4ae756c1 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:38.281925 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d9b68-3a5a-4edc-ad17-968f4ae756c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:38.282375 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d9b68-3a5a-4edc-ad17-968f4ae756c1 None admin] PATH_INFO: `/v3/users/fcafeca38d4942cf9f052bd972453739` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:38.283696 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d9b68-3a5a-4edc-ad17-968f4ae756c1 None admin] RBAC: Authorizing `identity:get_user(user_id=fcafeca38d4942cf9f052bd972453739)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:38.285374 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d9b68-3a5a-4edc-ad17-968f4ae756c1 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:38.287136 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 140/265] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:38 2022] GET /identity/v3/users/fcafeca38d4942cf9f052bd972453739 => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:46:38.293391 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d9b68-3a5a-4edc-ad17-968f4ae756c1 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:38.303058 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:38.395722 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d9b68-3a5a-4edc-ad17-968f4ae756c1 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:38.430872 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67514578-8d2d-4513-a977-dd8176051f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:38.431787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67514578-8d2d-4513-a977-dd8176051f70 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:38.432246 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67514578-8d2d-4513-a977-dd8176051f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:38.432656 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67514578-8d2d-4513-a977-dd8176051f70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:38.460534 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aede50-cd53-450b-bce5-dbc4f85c7c3e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:38.504900 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67514578-8d2d-4513-a977-dd8176051f70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:38.506617 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67514578-8d2d-4513-a977-dd8176051f70 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:38.509208 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:38.535264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643cac3-fdd9-4725-9052-9a074c12c88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:38.537106 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643cac3-fdd9-4725-9052-9a074c12c88b None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:38.537393 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643cac3-fdd9-4725-9052-9a074c12c88b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:38.537666 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643cac3-fdd9-4725-9052-9a074c12c88b None admin] PATH_INFO: `/v3/projects/e13bca418ede487ab7d1230e28403173/users/37f9da3b9a924300acf8f151691f0feb/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:38.544837 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643cac3-fdd9-4725-9052-9a074c12c88b None admin] RBAC: Authorizing `identity:create_grant(project_id=e13bca418ede487ab7d1230e28403173, user_id=37f9da3b9a924300acf8f151691f0feb, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:38.546333 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643cac3-fdd9-4725-9052-9a074c12c88b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:38.611734 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 141/266] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:46:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:46:38.617630 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67514578-8d2d-4513-a977-dd8176051f70 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:38.638406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 126/267] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Sep 9 20:46:38 2022] PUT /identity/v3/projects/e13bca418ede487ab7d1230e28403173/users/37f9da3b9a924300acf8f151691f0feb/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:46:38.780472 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67514578-8d2d-4513-a977-dd8176051f70 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:38.782408 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:38.818200 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df1d48-7907-48e5-99e3-d32db7169a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:38.818844 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df1d48-7907-48e5-99e3-d32db7169a8a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:38.819117 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df1d48-7907-48e5-99e3-d32db7169a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:38.819381 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df1d48-7907-48e5-99e3-d32db7169a8a None admin] PATH_INFO: `/v3/projects/e13bca418ede487ab7d1230e28403173` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:38.820555 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df1d48-7907-48e5-99e3-d32db7169a8a None admin] RBAC: Authorizing `identity:get_project(project_id=e13bca418ede487ab7d1230e28403173)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:38.822661 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df1d48-7907-48e5-99e3-d32db7169a8a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:38.824680 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 142/268] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:38 2022] GET /identity/v3/projects/e13bca418ede487ab7d1230e28403173 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:46:38.830658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643cac3-fdd9-4725-9052-9a074c12c88b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:38.870709 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:38.952342 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643cac3-fdd9-4725-9052-9a074c12c88b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:38.993698 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cbf3a2-c2a0-478d-9be5-e44cd56cef09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:38.994304 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbf3a2-c2a0-478d-9be5-e44cd56cef09 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:38.994583 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbf3a2-c2a0-478d-9be5-e44cd56cef09 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:38.994848 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbf3a2-c2a0-478d-9be5-e44cd56cef09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:38.995573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cbf3a2-c2a0-478d-9be5-e44cd56cef09 None admin] RBAC: Adding query filter params (role.id=13072ad4b81649459f59b8137db492ec, scope.project.id=e13bca418ede487ab7d1230e28403173, user.id=fcafeca38d4942cf9f052bd972453739) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:38.995878 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cbf3a2-c2a0-478d-9be5-e44cd56cef09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:38.997354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cbf3a2-c2a0-478d-9be5-e44cd56cef09 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:39.004366 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 127/269] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:46:38 2022] GET /identity/v3/role_assignments?user.id=fcafeca38d4942cf9f052bd972453739&scope.project.id=e13bca418ede487ab7d1230e28403173&role.id=13072ad4b81649459f59b8137db492ec => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:39.167827 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5c251-6876-41af-b1bd-2808704a51d9 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:39.168873 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5c251-6876-41af-b1bd-2808704a51d9 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:39.169369 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5c251-6876-41af-b1bd-2808704a51d9 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:39.170944 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 143/270] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:39.176860 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc1fc5-ae74-419a-9b4b-263d8972f73b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:39.177457 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc1fc5-ae74-419a-9b4b-263d8972f73b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:39.177929 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc1fc5-ae74-419a-9b4b-263d8972f73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:39.198332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-12bc1fc5-ae74-419a-9b4b-263d8972f73b None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:39.200826 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-12bc1fc5-ae74-419a-9b4b-263d8972f73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:39.290876 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 128/271] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:39.530665 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad5847-c608-479a-b960-35b49c54157b None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:39.531301 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad5847-c608-479a-b960-35b49c54157b None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:39.531800 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad5847-c608-479a-b960-35b49c54157b None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:39.533381 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 144/272] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:39.541974 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f4720-28df-4066-86ee-ffa97f886fff None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:39.542253 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f4720-28df-4066-86ee-ffa97f886fff None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:39.542527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f4720-28df-4066-86ee-ffa97f886fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:39.564920 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-094f4720-28df-4066-86ee-ffa97f886fff None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:39.566130 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-094f4720-28df-4066-86ee-ffa97f886fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:39.580579 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad5847-c608-479a-b960-35b49c54157b None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:39.586889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:39.646954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 129/273] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:39.701576 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad5847-c608-479a-b960-35b49c54157b None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:39.736950 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e11447-2175-49cf-b1c1-8997148be6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:39.737580 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e11447-2175-49cf-b1c1-8997148be6d8 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:39.737860 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e11447-2175-49cf-b1c1-8997148be6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:39.738130 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e11447-2175-49cf-b1c1-8997148be6d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:39.748549 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e11447-2175-49cf-b1c1-8997148be6d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:39.753486 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-f8e11447-2175-49cf-b1c1-8997148be6d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:39.753486 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:39.753486 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:39.753486 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:39.753486 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:39.753486 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:39.753486 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:39.753486 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:39.753486 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:39.753486 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:39.753486 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:39.753486 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:39.753486 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:39.753486 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:39.753486 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:39.753486 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:39.754370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:39.754370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:39.754370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:39.754370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:39.754370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:39.754370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:39.754370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:39.754370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:39.754370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:39.754370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:39.754370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:39.754370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:39.754370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:39.754370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:39.754370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:39.754370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:39.755159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:39.755159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:39.755159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:39.755159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:39.755159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:39.755159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:39.755159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:39.755159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:39.755159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:39.755159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:39.755159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:39.755159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:46:39.755159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 145/274] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:39.765695 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094f4720-28df-4066-86ee-ffa97f886fff None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:39.877707 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094f4720-28df-4066-86ee-ffa97f886fff None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:39.913224 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c5495-80bd-49a5-9f0c-1be9e74d8d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:39.913833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c5495-80bd-49a5-9f0c-1be9e74d8d7b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:39.914116 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c5495-80bd-49a5-9f0c-1be9e74d8d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:39.914977 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c5495-80bd-49a5-9f0c-1be9e74d8d7b None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:39.915621 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c5495-80bd-49a5-9f0c-1be9e74d8d7b None admin] RBAC: Adding query filter params (name=member) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:39.915932 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c5495-80bd-49a5-9f0c-1be9e74d8d7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:39.917353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c5495-80bd-49a5-9f0c-1be9e74d8d7b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:39.926354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 130/275] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Sep 9 20:46:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:39.931054 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e11447-2175-49cf-b1c1-8997148be6d8 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:39.932030 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c5495-80bd-49a5-9f0c-1be9e74d8d7b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:39.954030 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:39.964129 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:40.048319 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e11447-2175-49cf-b1c1-8997148be6d8 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:40.048536 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c5495-80bd-49a5-9f0c-1be9e74d8d7b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:40.083072 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f3899-5082-4a6c-ab33-a81825037711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:40.083705 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f3899-5082-4a6c-ab33-a81825037711 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:40.083985 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f3899-5082-4a6c-ab33-a81825037711 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:40.084286 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f3899-5082-4a6c-ab33-a81825037711 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:40.085161 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d83b80-534b-4ee1-a843-cb852da109b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:40.085393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f3899-5082-4a6c-ab33-a81825037711 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:40.085784 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d83b80-534b-4ee1-a843-cb852da109b6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:40.086060 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d83b80-534b-4ee1-a843-cb852da109b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:40.087039 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d83b80-534b-4ee1-a843-cb852da109b6 None admin] PATH_INFO: `/v3/users/4d397df6246d4faebcfe0da3225afe0f` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:40.087470 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f3899-5082-4a6c-ab33-a81825037711 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:40.089243 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 146/276] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:46:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:40.094630 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c557f-0f34-44f3-8bab-c2c0df67883c None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:40.094921 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c557f-0f34-44f3-8bab-c2c0df67883c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:40.095203 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c557f-0f34-44f3-8bab-c2c0df67883c None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:40.096482 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 147/277] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:40.100412 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c557f-0f34-44f3-8bab-c2c0df67883c None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:40.105108 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d83b80-534b-4ee1-a843-cb852da109b6 None admin] RBAC: Authorizing `identity:get_user(user_id=4d397df6246d4faebcfe0da3225afe0f)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:40.106586 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d83b80-534b-4ee1-a843-cb852da109b6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:40.108272 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 131/278] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:39 2022] GET /identity/v3/users/4d397df6246d4faebcfe0da3225afe0f => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:46:40.109422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f3894-aa61-418d-b310-d7759b7675dc None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:40.109701 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f3894-aa61-418d-b310-d7759b7675dc None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:40.109975 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f3894-aa61-418d-b310-d7759b7675dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:40.127362 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f52f3894-aa61-418d-b310-d7759b7675dc None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:40.132177 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f52f3894-aa61-418d-b310-d7759b7675dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:40.200446 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 132/279] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:40.201298 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f3894-aa61-418d-b310-d7759b7675dc None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:40.204742 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:40.206096 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:40.216472 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c557f-0f34-44f3-8bab-c2c0df67883c None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:40.257958 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861773c-254d-4b0a-9b94-d01586775abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:40.259195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861773c-254d-4b0a-9b94-d01586775abd None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:40.259475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861773c-254d-4b0a-9b94-d01586775abd None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:40.259748 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861773c-254d-4b0a-9b94-d01586775abd None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:40.260466 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861773c-254d-4b0a-9b94-d01586775abd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:40.262948 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861773c-254d-4b0a-9b94-d01586775abd None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:40.323883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f3894-aa61-418d-b310-d7759b7675dc None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:40.358233 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46096b-e2c2-4c1d-a107-5cd9abbcd7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:40.358839 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46096b-e2c2-4c1d-a107-5cd9abbcd7bc None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:40.359168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46096b-e2c2-4c1d-a107-5cd9abbcd7bc None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:40.359456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46096b-e2c2-4c1d-a107-5cd9abbcd7bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:40.360195 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb46096b-e2c2-4c1d-a107-5cd9abbcd7bc None admin] RBAC: Adding query filter params (role.id=c59d891d042e48bab625f7b3666ef536, user.id=4d397df6246d4faebcfe0da3225afe0f, scope.system=all) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:40.360505 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb46096b-e2c2-4c1d-a107-5cd9abbcd7bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:40.361941 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb46096b-e2c2-4c1d-a107-5cd9abbcd7bc None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:40.376103 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 133/280] 10.222.0.48 () {56 vars in 1308 bytes} [Fri Sep 9 20:46:40 2022] GET /identity/v3/role_assignments?user.id=4d397df6246d4faebcfe0da3225afe0f&scope.system=all&role.id=c59d891d042e48bab625f7b3666ef536 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:40.380602 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 148/281] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Sep 9 20:46:40 2022] POST /identity/v3/users => generated 293 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:46:40.471081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46096b-e2c2-4c1d-a107-5cd9abbcd7bc None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:40.550440 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:40.601464 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46096b-e2c2-4c1d-a107-5cd9abbcd7bc None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:40.640588 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3dfc73-7d73-4921-b8d4-432762e84021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:40.641235 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3dfc73-7d73-4921-b8d4-432762e84021 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:40.641516 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3dfc73-7d73-4921-b8d4-432762e84021 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:40.641788 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3dfc73-7d73-4921-b8d4-432762e84021 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:40.653142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3dfc73-7d73-4921-b8d4-432762e84021 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:40.658207 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-ac3dfc73-7d73-4921-b8d4-432762e84021 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:46:40.658207 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:40.658207 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:40.658207 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:40.658207 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:40.658207 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:40.658207 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:40.658207 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:40.658207 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:40.658207 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:40.658207 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:40.658207 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:40.658207 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:40.658207 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:40.658207 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:40.658207 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:40.659076 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:40.659076 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:40.659076 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:40.659076 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:40.659076 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:40.659076 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:40.659076 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:40.659076 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:40.659076 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:40.659076 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:40.659076 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:40.659076 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:40.659076 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:40.659076 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:40.659076 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:40.659076 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:40.660578 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:40.660578 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:40.660578 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:40.660578 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:40.660578 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:40.660578 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:40.660578 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:40.660578 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:40.660578 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:40.660578 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:40.660578 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:46:40.660578 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:46:40.660578 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 134/282] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:40.665302 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861773c-254d-4b0a-9b94-d01586775abd None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:40.787250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861773c-254d-4b0a-9b94-d01586775abd None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:40.826685 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f04d9-7d90-49fa-b16a-a93a538ccfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:40.827270 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f04d9-7d90-49fa-b16a-a93a538ccfdf None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:40.827516 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f04d9-7d90-49fa-b16a-a93a538ccfdf None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:40.827781 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f04d9-7d90-49fa-b16a-a93a538ccfdf None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:40.828488 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f04d9-7d90-49fa-b16a-a93a538ccfdf None admin] RBAC: Adding query filter params (name=reader) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:40.828799 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f04d9-7d90-49fa-b16a-a93a538ccfdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:40.830166 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f04d9-7d90-49fa-b16a-a93a538ccfdf None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:40.840193 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 149/283] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Sep 9 20:46:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:40.845615 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3dfc73-7d73-4921-b8d4-432762e84021 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:40.972414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3dfc73-7d73-4921-b8d4-432762e84021 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:41.002571 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:41.016704 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff870886-6d1b-4628-abbb-dd2710058198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:41.021132 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff870886-6d1b-4628-abbb-dd2710058198 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:41.021132 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff870886-6d1b-4628-abbb-dd2710058198 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:41.021132 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff870886-6d1b-4628-abbb-dd2710058198 None admin] PATH_INFO: `/v3/users/ccf3f53545074574bc218dad082d70cc` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:41.025118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff870886-6d1b-4628-abbb-dd2710058198 None admin] RBAC: Authorizing `identity:get_user(user_id=ccf3f53545074574bc218dad082d70cc)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:41.026639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff870886-6d1b-4628-abbb-dd2710058198 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:41.028813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 135/284] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:40 2022] GET /identity/v3/users/ccf3f53545074574bc218dad082d70cc => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:41.034675 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f04d9-7d90-49fa-b16a-a93a538ccfdf None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:41.071303 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:41.146843 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83530e5c-d6bc-442b-b229-dccc03a50f2a None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:41.147203 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83530e5c-d6bc-442b-b229-dccc03a50f2a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:41.147393 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83530e5c-d6bc-442b-b229-dccc03a50f2a None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:41.149598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 136/285] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:46:41.154556 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a56bc8-e494-4212-b10c-1ac8408f428a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:41.155080 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a56bc8-e494-4212-b10c-1ac8408f428a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:41.155556 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a56bc8-e494-4212-b10c-1ac8408f428a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:41.193658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b2a56bc8-e494-4212-b10c-1ac8408f428a None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:41.195226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b2a56bc8-e494-4212-b10c-1ac8408f428a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:41.205287 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f04d9-7d90-49fa-b16a-a93a538ccfdf None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:41.249924 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27471d17-4472-45fc-8ab7-afc55f274601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:41.251380 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27471d17-4472-45fc-8ab7-afc55f274601 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:41.252661 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27471d17-4472-45fc-8ab7-afc55f274601 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:41.253142 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27471d17-4472-45fc-8ab7-afc55f274601 None admin] PATH_INFO: `/v3/projects/e13bca418ede487ab7d1230e28403173` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:41.254577 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27471d17-4472-45fc-8ab7-afc55f274601 None admin] RBAC: Authorizing `identity:get_project(project_id=e13bca418ede487ab7d1230e28403173)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:41.256842 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27471d17-4472-45fc-8ab7-afc55f274601 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:41.260911 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 150/286] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:41 2022] GET /identity/v3/projects/e13bca418ede487ab7d1230e28403173 => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:41.265434 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27471d17-4472-45fc-8ab7-afc55f274601 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:41.334904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 137/287] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:46:41.407625 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:41.420386 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27471d17-4472-45fc-8ab7-afc55f274601 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:41.475268 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79ff739-9301-4fe3-b9b5-99fb38b87ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:41.476393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ff739-9301-4fe3-b9b5-99fb38b87ddd None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:41.476963 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ff739-9301-4fe3-b9b5-99fb38b87ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:41.478001 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ff739-9301-4fe3-b9b5-99fb38b87ddd None admin] PATH_INFO: `/v3/projects/e13bca418ede487ab7d1230e28403173/users/ccf3f53545074574bc218dad082d70cc/roles/e01870d6deef4f1ebbc7470c342046a6` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:41.485422 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79ff739-9301-4fe3-b9b5-99fb38b87ddd None admin] RBAC: Authorizing `identity:create_grant(project_id=e13bca418ede487ab7d1230e28403173, user_id=ccf3f53545074574bc218dad082d70cc, role_id=e01870d6deef4f1ebbc7470c342046a6)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:41.487194 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79ff739-9301-4fe3-b9b5-99fb38b87ddd None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:41.572056 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 151/288] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Sep 9 20:46:41 2022] PUT /identity/v3/projects/e13bca418ede487ab7d1230e28403173/users/ccf3f53545074574bc218dad082d70cc/roles/e01870d6deef4f1ebbc7470c342046a6 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:46:41.710690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a56bc8-e494-4212-b10c-1ac8408f428a None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:41.843406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a56bc8-e494-4212-b10c-1ac8408f428a None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:41.871197 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:41.881671 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369076ec-6901-42c9-9db8-04e3befea959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:41.882564 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369076ec-6901-42c9-9db8-04e3befea959 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:41.885690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369076ec-6901-42c9-9db8-04e3befea959 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:41.885690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369076ec-6901-42c9-9db8-04e3befea959 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:41.896678 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369076ec-6901-42c9-9db8-04e3befea959 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:41.901912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-369076ec-6901-42c9-9db8-04e3befea959 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:41.901912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:41.901912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:41.901912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:41.901912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:41.901912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:41.901912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:41.901912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:41.901912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:41.901912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:41.901912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:41.901912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:41.901912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:41.901912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:41.901912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:41.901912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:41.902809 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:41.902809 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:41.902809 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:41.902809 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:41.902809 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:41.902809 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:41.902809 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:41.902809 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:41.902809 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:41.902809 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:41.902809 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:41.902809 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:41.902809 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:41.902809 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:41.902809 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:41.902809 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:41.903619 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:41.903619 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:41.903619 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:41.903619 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:41.903619 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:41.903619 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:41.903619 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:41.903619 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:41.903619 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:41.903619 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:41.903619 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:41.903619 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:46:41.905810 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 138/289] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:41.911777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79ff739-9301-4fe3-b9b5-99fb38b87ddd None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:42.049943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79ff739-9301-4fe3-b9b5-99fb38b87ddd None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:42.093286 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44be65-eab8-433e-a056-e6667a407fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:42.094169 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44be65-eab8-433e-a056-e6667a407fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:42.094598 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44be65-eab8-433e-a056-e6667a407fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:42.095031 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44be65-eab8-433e-a056-e6667a407fc1 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:42.095841 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44be65-eab8-433e-a056-e6667a407fc1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:42.096328 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44be65-eab8-433e-a056-e6667a407fc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:42.097860 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44be65-eab8-433e-a056-e6667a407fc1 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:42.106705 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 152/290] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Sep 9 20:46:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:42.112656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369076ec-6901-42c9-9db8-04e3befea959 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:42.240569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369076ec-6901-42c9-9db8-04e3befea959 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:42.282343 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7040f02a-6993-4a5a-b32e-d8be338c5862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:42.283225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7040f02a-6993-4a5a-b32e-d8be338c5862 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:42.283706 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7040f02a-6993-4a5a-b32e-d8be338c5862 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:42.284148 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7040f02a-6993-4a5a-b32e-d8be338c5862 None admin] PATH_INFO: `/v3/users/37f9da3b9a924300acf8f151691f0feb` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:42.288735 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7040f02a-6993-4a5a-b32e-d8be338c5862 None admin] RBAC: Authorizing `identity:get_user(user_id=37f9da3b9a924300acf8f151691f0feb)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:42.290892 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7040f02a-6993-4a5a-b32e-d8be338c5862 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:42.292663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 139/291] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:42 2022] GET /identity/v3/users/37f9da3b9a924300acf8f151691f0feb => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:42.298399 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44be65-eab8-433e-a056-e6667a407fc1 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:42.312093 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:42.422266 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44be65-eab8-433e-a056-e6667a407fc1 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:42.469973 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d970f-fb24-4712-9db9-d059d86efb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:42.470863 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d970f-fb24-4712-9db9-d059d86efb0e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:42.471289 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d970f-fb24-4712-9db9-d059d86efb0e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:42.471706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d970f-fb24-4712-9db9-d059d86efb0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:42.563470 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81d970f-fb24-4712-9db9-d059d86efb0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:42.565100 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81d970f-fb24-4712-9db9-d059d86efb0e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:42.576385 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:42.697259 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 153/292] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:46:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:46:42.702888 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7040f02a-6993-4a5a-b32e-d8be338c5862 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:42.780052 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:42.837471 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7040f02a-6993-4a5a-b32e-d8be338c5862 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:42.875696 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f81520-298a-461a-bde9-6b70f5de6600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:42.876379 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f81520-298a-461a-bde9-6b70f5de6600 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:42.876662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f81520-298a-461a-bde9-6b70f5de6600 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:42.876937 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f81520-298a-461a-bde9-6b70f5de6600 None admin] PATH_INFO: `/v3/projects/e13bca418ede487ab7d1230e28403173` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:42.878083 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f81520-298a-461a-bde9-6b70f5de6600 None admin] RBAC: Authorizing `identity:get_project(project_id=e13bca418ede487ab7d1230e28403173)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:42.879618 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f81520-298a-461a-bde9-6b70f5de6600 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:42.881605 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 140/293] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:42 2022] GET /identity/v3/projects/e13bca418ede487ab7d1230e28403173 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:42.887431 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d970f-fb24-4712-9db9-d059d86efb0e None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:42.997666 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b67a9-1336-43e3-a07a-864caae634b8 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:42.998066 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b67a9-1336-43e3-a07a-864caae634b8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:42.998346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b67a9-1336-43e3-a07a-864caae634b8 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:42.999938 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 141/294] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:43.005759 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a1db9-0062-40d2-9541-a1537e924be1 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:43.006046 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a1db9-0062-40d2-9541-a1537e924be1 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:43.006320 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a1db9-0062-40d2-9541-a1537e924be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:43.035207 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d970f-fb24-4712-9db9-d059d86efb0e None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:43.037204 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-106a1db9-0062-40d2-9541-a1537e924be1 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:43.038528 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-106a1db9-0062-40d2-9541-a1537e924be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:43.042816 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:43.088313 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87eb69-8268-41a2-92c0-ce15b483f0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:43.088855 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87eb69-8268-41a2-92c0-ce15b483f0cd None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:43.089131 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87eb69-8268-41a2-92c0-ce15b483f0cd None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:43.089397 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87eb69-8268-41a2-92c0-ce15b483f0cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:43.090146 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be87eb69-8268-41a2-92c0-ce15b483f0cd None admin] RBAC: Adding query filter params (role.id=c59d891d042e48bab625f7b3666ef536, scope.project.id=e13bca418ede487ab7d1230e28403173, user.id=37f9da3b9a924300acf8f151691f0feb) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:43.090452 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be87eb69-8268-41a2-92c0-ce15b483f0cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:43.091942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be87eb69-8268-41a2-92c0-ce15b483f0cd None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:43.108698 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 154/295] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:46:42 2022] GET /identity/v3/role_assignments?user.id=37f9da3b9a924300acf8f151691f0feb&scope.project.id=e13bca418ede487ab7d1230e28403173&role.id=c59d891d042e48bab625f7b3666ef536 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:43.158455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 142/296] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:43.255707 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35b8d1-086a-41a5-b882-78fe4ec18308 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:43.256364 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35b8d1-086a-41a5-b882-78fe4ec18308 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:43.256833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35b8d1-086a-41a5-b882-78fe4ec18308 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:43.258964 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 155/297] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:43.269491 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b9b3b-0bb6-480a-8c94-1713fd01a5c2 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:43.270117 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b9b3b-0bb6-480a-8c94-1713fd01a5c2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:43.270582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b9b3b-0bb6-480a-8c94-1713fd01a5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:43.296378 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b61b9b3b-0bb6-480a-8c94-1713fd01a5c2 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:43.297575 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b61b9b3b-0bb6-480a-8c94-1713fd01a5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:43.348638 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:43.357893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f91a5-9db5-4d43-a854-a6389acfe69d None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:43.358765 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f91a5-9db5-4d43-a854-a6389acfe69d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:43.359192 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f91a5-9db5-4d43-a854-a6389acfe69d None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:43.360896 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 156/298] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:43.366659 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969825b-6541-48c8-92d7-37c99de560f4 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:43.366935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969825b-6541-48c8-92d7-37c99de560f4 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:43.367207 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969825b-6541-48c8-92d7-37c99de560f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:43.370345 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 143/299] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:46:43.395621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-5969825b-6541-48c8-92d7-37c99de560f4 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:43.397088 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5969825b-6541-48c8-92d7-37c99de560f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:43.466848 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 157/300] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:46:43.523822 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61b9b3b-0bb6-480a-8c94-1713fd01a5c2 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:43.638693 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61b9b3b-0bb6-480a-8c94-1713fd01a5c2 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:43.658619 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969825b-6541-48c8-92d7-37c99de560f4 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:43.681164 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a9722b-fb10-4dcc-b6e5-8f54c8e79a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:43.682828 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9722b-fb10-4dcc-b6e5-8f54c8e79a62 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:43.683105 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9722b-fb10-4dcc-b6e5-8f54c8e79a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:43.683377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9722b-fb10-4dcc-b6e5-8f54c8e79a62 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:43.705514 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a9722b-fb10-4dcc-b6e5-8f54c8e79a62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:43.710393 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-e4a9722b-fb10-4dcc-b6e5-8f54c8e79a62 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:43.710393 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:43.710393 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:43.710393 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:43.710393 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:43.710393 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:43.710393 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:43.710393 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:43.710393 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:43.710393 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:43.710393 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:43.710393 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:43.710393 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:43.710393 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:43.710393 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:43.710393 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:43.711290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:43.711290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:43.711290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:43.711290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:43.711290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:43.711290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:43.711290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:43.711290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:43.711290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:43.711290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:43.711290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:43.711290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:43.711290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:43.711290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:43.711290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:43.711290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:43.712530 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:43.712530 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:43.712530 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:43.712530 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:43.712530 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:43.712530 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:43.712530 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:43.712530 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:43.712530 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:43.712530 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:43.712530 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:43.712530 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:46:43.712530 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 144/301] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:43.719640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a9722b-fb10-4dcc-b6e5-8f54c8e79a62 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:43.783377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:43.814729 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:43.833796 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969825b-6541-48c8-92d7-37c99de560f4 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:43.849963 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a9722b-fb10-4dcc-b6e5-8f54c8e79a62 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:43.861253 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84225492-ecfc-4daa-b60c-fef8f1be7743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:43.861896 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84225492-ecfc-4daa-b60c-fef8f1be7743 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:43.862174 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84225492-ecfc-4daa-b60c-fef8f1be7743 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:43.866614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84225492-ecfc-4daa-b60c-fef8f1be7743 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:43.866614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84225492-ecfc-4daa-b60c-fef8f1be7743 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:43.866614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84225492-ecfc-4daa-b60c-fef8f1be7743 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:43.867353 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 158/302] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:46:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:43.868885 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84225492-ecfc-4daa-b60c-fef8f1be7743 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:43.904268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a2a37-3951-45d6-a1e8-8cb47fa4fe2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:43.904892 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a2a37-3951-45d6-a1e8-8cb47fa4fe2d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:43.905379 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a2a37-3951-45d6-a1e8-8cb47fa4fe2d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:43.905654 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a2a37-3951-45d6-a1e8-8cb47fa4fe2d None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:43.906329 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a2a37-3951-45d6-a1e8-8cb47fa4fe2d None admin] RBAC: Adding query filter params (name=member) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:43.906636 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a2a37-3951-45d6-a1e8-8cb47fa4fe2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:43.908086 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a2a37-3951-45d6-a1e8-8cb47fa4fe2d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:43.925848 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 145/303] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Sep 9 20:46:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:43.925848 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a2a37-3951-45d6-a1e8-8cb47fa4fe2d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:44.004538 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84225492-ecfc-4daa-b60c-fef8f1be7743 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:44.050468 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7d87f-56e9-4adc-ad8d-66b1b89c1f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:44.051108 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7d87f-56e9-4adc-ad8d-66b1b89c1f92 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:44.051388 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7d87f-56e9-4adc-ad8d-66b1b89c1f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:44.051503 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a2a37-3951-45d6-a1e8-8cb47fa4fe2d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:44.051652 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7d87f-56e9-4adc-ad8d-66b1b89c1f92 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:44.054342 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:44.066586 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f7d87f-56e9-4adc-ad8d-66b1b89c1f92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:44.071859 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-14f7d87f-56e9-4adc-ad8d-66b1b89c1f92 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:46:44.071859 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:44.071859 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:44.071859 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:44.071859 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:44.071859 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:44.071859 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:44.071859 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:44.071859 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:44.071859 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:44.071859 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:44.071859 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:44.071859 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:44.071859 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:44.071859 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:44.071859 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:44.073404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:44.073404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:44.073404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:44.073404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:44.073404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:44.073404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:44.073404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:44.073404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:44.073404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:44.073404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:44.073404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:44.073404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:44.073404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:44.073404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:44.073404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:44.073404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:44.074211 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:44.074211 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:44.074211 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:44.074211 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:44.074211 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:44.074211 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:44.074211 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:44.074211 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:44.074211 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:44.074211 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:44.074211 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:46:44.074211 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:46:44.077990 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 159/304] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:44.078955 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7d87f-56e9-4adc-ad8d-66b1b89c1f92 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:44.092937 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51ed9c-35a7-4f4b-9a5e-65127be7b8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:44.094012 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51ed9c-35a7-4f4b-9a5e-65127be7b8df None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:44.094398 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51ed9c-35a7-4f4b-9a5e-65127be7b8df None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:44.094781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51ed9c-35a7-4f4b-9a5e-65127be7b8df None admin] PATH_INFO: `/v3/groups` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:44.095623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51ed9c-35a7-4f4b-9a5e-65127be7b8df None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:44.097396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51ed9c-35a7-4f4b-9a5e-65127be7b8df None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:44.112339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:44.184501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7d87f-56e9-4adc-ad8d-66b1b89c1f92 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:44.210190 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 146/305] 10.222.0.48 () {60 vars in 1134 bytes} [Fri Sep 9 20:46:43 2022] POST /identity/v3/groups => generated 229 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:46:44.210902 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51ed9c-35a7-4f4b-9a5e-65127be7b8df None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:44.231341 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa1439f-c96c-4baa-a954-d7ea02ddf95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:44.232678 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1439f-c96c-4baa-a954-d7ea02ddf95a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:44.233125 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1439f-c96c-4baa-a954-d7ea02ddf95a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:44.233563 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1439f-c96c-4baa-a954-d7ea02ddf95a None admin] PATH_INFO: `/v3/users/4d397df6246d4faebcfe0da3225afe0f` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:44.238280 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa1439f-c96c-4baa-a954-d7ea02ddf95a None admin] RBAC: Authorizing `identity:get_user(user_id=4d397df6246d4faebcfe0da3225afe0f)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:44.238280 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa1439f-c96c-4baa-a954-d7ea02ddf95a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:44.240944 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 160/306] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:44 2022] GET /identity/v3/users/4d397df6246d4faebcfe0da3225afe0f => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:44.246344 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa1439f-c96c-4baa-a954-d7ea02ddf95a None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:44.341831 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51ed9c-35a7-4f4b-9a5e-65127be7b8df None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:44.366947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:44.374703 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa1439f-c96c-4baa-a954-d7ea02ddf95a None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:44.381192 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9c03d-d306-4024-9bcd-1bb9a020d4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:44.381952 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9c03d-d306-4024-9bcd-1bb9a020d4d9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:44.382294 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9c03d-d306-4024-9bcd-1bb9a020d4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:44.385551 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9c03d-d306-4024-9bcd-1bb9a020d4d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:44.385551 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a9c03d-d306-4024-9bcd-1bb9a020d4d9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:44.385551 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a9c03d-d306-4024-9bcd-1bb9a020d4d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:44.385551 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a9c03d-d306-4024-9bcd-1bb9a020d4d9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:44.395268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 147/307] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Sep 9 20:46:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:44.403710 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9c03d-d306-4024-9bcd-1bb9a020d4d9 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:44.423811 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3bf208-dd4a-44a9-acff-91b74cb63bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:44.424606 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bf208-dd4a-44a9-acff-91b74cb63bee None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:44.424885 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bf208-dd4a-44a9-acff-91b74cb63bee None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:44.425156 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bf208-dd4a-44a9-acff-91b74cb63bee None admin] PATH_INFO: `/v3/system/users/4d397df6246d4faebcfe0da3225afe0f/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:44.426663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3bf208-dd4a-44a9-acff-91b74cb63bee None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4d397df6246d4faebcfe0da3225afe0f, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:44.429149 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3bf208-dd4a-44a9-acff-91b74cb63bee None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:44.444511 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:44.515803 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9c03d-d306-4024-9bcd-1bb9a020d4d9 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:44.526490 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 161/308] 10.222.0.48 () {58 vars in 1334 bytes} [Fri Sep 9 20:46:44 2022] PUT /identity/v3/system/users/4d397df6246d4faebcfe0da3225afe0f/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:46:44.527935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bfb26-6d29-4642-adc4-dbae99f40656 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:44.528249 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bfb26-6d29-4642-adc4-dbae99f40656 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:44.528525 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bfb26-6d29-4642-adc4-dbae99f40656 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:44.529932 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 162/309] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:44.535730 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce6b59-5bd4-4ea4-a3ed-50ec9f8f2aca None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:44.536006 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce6b59-5bd4-4ea4-a3ed-50ec9f8f2aca None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:44.539738 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce6b59-5bd4-4ea4-a3ed-50ec9f8f2aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:44.558218 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01740bd2-e26f-4aa6-b012-d1776dbdcfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:44.559235 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01740bd2-e26f-4aa6-b012-d1776dbdcfdc None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:44.559954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01740bd2-e26f-4aa6-b012-d1776dbdcfdc None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:44.560693 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01740bd2-e26f-4aa6-b012-d1776dbdcfdc None admin] PATH_INFO: `/v3/users/b9270c31ba524eea9bb7ca9ae2c00a5e` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:44.567317 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-83ce6b59-5bd4-4ea4-a3ed-50ec9f8f2aca None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:44.568639 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-83ce6b59-5bd4-4ea4-a3ed-50ec9f8f2aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:44.579036 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01740bd2-e26f-4aa6-b012-d1776dbdcfdc None admin] RBAC: Authorizing `identity:get_user(user_id=b9270c31ba524eea9bb7ca9ae2c00a5e)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:44.580945 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01740bd2-e26f-4aa6-b012-d1776dbdcfdc None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:44.583385 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 148/310] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:44 2022] GET /identity/v3/users/b9270c31ba524eea9bb7ca9ae2c00a5e => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:44.590314 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01740bd2-e26f-4aa6-b012-d1776dbdcfdc None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:44.657257 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 163/311] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:44.748389 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:44.765119 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01740bd2-e26f-4aa6-b012-d1776dbdcfdc None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:44.799341 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707dfffa-de5e-440a-bc05-f16882e0c5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:44.799948 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707dfffa-de5e-440a-bc05-f16882e0c5d7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:44.800256 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707dfffa-de5e-440a-bc05-f16882e0c5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:44.800531 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707dfffa-de5e-440a-bc05-f16882e0c5d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:44.801320 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707dfffa-de5e-440a-bc05-f16882e0c5d7 None admin] RBAC: Adding query filter params (role.id=e01870d6deef4f1ebbc7470c342046a6, user.id=b9270c31ba524eea9bb7ca9ae2c00a5e, scope.system=all) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:44.801632 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707dfffa-de5e-440a-bc05-f16882e0c5d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:44.803118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707dfffa-de5e-440a-bc05-f16882e0c5d7 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:44.808600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 149/312] 10.222.0.48 () {56 vars in 1308 bytes} [Fri Sep 9 20:46:44 2022] GET /identity/v3/role_assignments?user.id=b9270c31ba524eea9bb7ca9ae2c00a5e&scope.system=all&role.id=e01870d6deef4f1ebbc7470c342046a6 => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:46:44.928453 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce6b59-5bd4-4ea4-a3ed-50ec9f8f2aca None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:45.029662 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:45.039377 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce6b59-5bd4-4ea4-a3ed-50ec9f8f2aca None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:45.075760 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6104bc-3f85-4a0f-97b0-0cb0bb0eb661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:45.076671 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6104bc-3f85-4a0f-97b0-0cb0bb0eb661 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:45.077095 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6104bc-3f85-4a0f-97b0-0cb0bb0eb661 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:45.077569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6104bc-3f85-4a0f-97b0-0cb0bb0eb661 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:45.088272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6104bc-3f85-4a0f-97b0-0cb0bb0eb661 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:45.093182 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-2c6104bc-3f85-4a0f-97b0-0cb0bb0eb661 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:46:45.093182 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:45.093182 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:45.093182 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:45.093182 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:45.093182 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:45.093182 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:45.093182 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:45.093182 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:45.093182 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:45.093182 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:45.093182 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:45.093182 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:45.093182 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:45.093182 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:45.093182 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:45.094052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:45.094052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:45.094052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:45.094052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:45.094052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:45.094052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:45.094052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:45.094052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:45.094052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:45.094052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:45.094052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:45.094052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:45.094052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:45.094052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:45.094052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:45.094052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:45.094851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:45.094851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:45.094851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:45.094851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:45.094851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:45.094851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:45.094851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:45.094851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:45.094851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:45.094851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:45.094851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:46:45.094851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:46:45.094851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 164/313] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:45.101462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707dfffa-de5e-440a-bc05-f16882e0c5d7 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:45.210643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707dfffa-de5e-440a-bc05-f16882e0c5d7 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:45.246420 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b808a9e-dfb7-46a9-985c-d70ef969aa37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:45.247373 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b808a9e-dfb7-46a9-985c-d70ef969aa37 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:45.247864 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b808a9e-dfb7-46a9-985c-d70ef969aa37 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:45.248355 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b808a9e-dfb7-46a9-985c-d70ef969aa37 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:45.249239 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b808a9e-dfb7-46a9-985c-d70ef969aa37 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:45.249734 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b808a9e-dfb7-46a9-985c-d70ef969aa37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:45.251397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b808a9e-dfb7-46a9-985c-d70ef969aa37 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:45.260400 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 150/314] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Sep 9 20:46:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:45.271284 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6104bc-3f85-4a0f-97b0-0cb0bb0eb661 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:45.383866 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6104bc-3f85-4a0f-97b0-0cb0bb0eb661 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:45.418693 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef9d0a-0fd5-4c71-bf1a-51df6947c51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:45.419331 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef9d0a-0fd5-4c71-bf1a-51df6947c51e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:45.419611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef9d0a-0fd5-4c71-bf1a-51df6947c51e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:45.419883 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef9d0a-0fd5-4c71-bf1a-51df6947c51e None admin] PATH_INFO: `/v3/users/ccf3f53545074574bc218dad082d70cc` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:45.421142 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef9d0a-0fd5-4c71-bf1a-51df6947c51e None admin] RBAC: Authorizing `identity:get_user(user_id=ccf3f53545074574bc218dad082d70cc)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:45.422648 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef9d0a-0fd5-4c71-bf1a-51df6947c51e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:45.424589 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 165/315] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:45 2022] GET /identity/v3/users/ccf3f53545074574bc218dad082d70cc => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:45.430714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b808a9e-dfb7-46a9-985c-d70ef969aa37 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:45.444228 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:45.535366 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b808a9e-dfb7-46a9-985c-d70ef969aa37 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:45.570892 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32951e1b-0b08-46bc-a6a6-d8efc3e009ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:45.571858 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32951e1b-0b08-46bc-a6a6-d8efc3e009ef None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:45.572374 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32951e1b-0b08-46bc-a6a6-d8efc3e009ef None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:45.572838 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32951e1b-0b08-46bc-a6a6-d8efc3e009ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:45.643938 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32951e1b-0b08-46bc-a6a6-d8efc3e009ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:45.645754 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32951e1b-0b08-46bc-a6a6-d8efc3e009ef None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:45.657977 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:45.746377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 151/316] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:46:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:46:45.751708 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef9d0a-0fd5-4c71-bf1a-51df6947c51e None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:45.771723 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:45.862281 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef9d0a-0fd5-4c71-bf1a-51df6947c51e None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:45.897042 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ed422-da73-40b7-ab9d-9aa51717dcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:45.898031 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ed422-da73-40b7-ab9d-9aa51717dcad None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:45.898506 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ed422-da73-40b7-ab9d-9aa51717dcad None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:45.899505 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ed422-da73-40b7-ab9d-9aa51717dcad None admin] PATH_INFO: `/v3/projects/e13bca418ede487ab7d1230e28403173` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:45.900881 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ed422-da73-40b7-ab9d-9aa51717dcad None admin] RBAC: Authorizing `identity:get_project(project_id=e13bca418ede487ab7d1230e28403173)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:45.902579 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ed422-da73-40b7-ab9d-9aa51717dcad None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:45.904741 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 166/317] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:45 2022] GET /identity/v3/projects/e13bca418ede487ab7d1230e28403173 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:45.915564 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32951e1b-0b08-46bc-a6a6-d8efc3e009ef None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:46.016863 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32951e1b-0b08-46bc-a6a6-d8efc3e009ef None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:46.028334 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:46.051430 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d7d4c-485d-47fe-b627-694bb32e48b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:46.052035 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d7d4c-485d-47fe-b627-694bb32e48b6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:46.054964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d7d4c-485d-47fe-b627-694bb32e48b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:46.054964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d7d4c-485d-47fe-b627-694bb32e48b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:46.054964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d7d4c-485d-47fe-b627-694bb32e48b6 None admin] RBAC: Adding query filter params (role.id=e01870d6deef4f1ebbc7470c342046a6, scope.project.id=e13bca418ede487ab7d1230e28403173, user.id=ccf3f53545074574bc218dad082d70cc) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:46.054964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d7d4c-485d-47fe-b627-694bb32e48b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:46.055249 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d7d4c-485d-47fe-b627-694bb32e48b6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:46.061347 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 152/318] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:46:45 2022] GET /identity/v3/role_assignments?user.id=ccf3f53545074574bc218dad082d70cc&scope.project.id=e13bca418ede487ab7d1230e28403173&role.id=e01870d6deef4f1ebbc7470c342046a6 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:46.520800 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7dba8-4d51-4bad-a689-c1dbd9652e9e None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:46.521360 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7dba8-4d51-4bad-a689-c1dbd9652e9e None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:46.521360 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7dba8-4d51-4bad-a689-c1dbd9652e9e None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:46.522818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 167/319] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:46.527829 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7a542-c5cd-4015-a7fa-b6a5e8147ce0 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:46.528151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7a542-c5cd-4015-a7fa-b6a5e8147ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:46.528428 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7a542-c5cd-4015-a7fa-b6a5e8147ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:46.546886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a9f7a542-c5cd-4015-a7fa-b6a5e8147ce0 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:46.548099 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a9f7a542-c5cd-4015-a7fa-b6a5e8147ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:46.614537 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 153/320] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:46.752773 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd7a77-b72c-4d2f-8ef3-69e66c0ca910 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:46.753126 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd7a77-b72c-4d2f-8ef3-69e66c0ca910 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:46.753329 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd7a77-b72c-4d2f-8ef3-69e66c0ca910 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:46.754803 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 168/321] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:46.759964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04748eef-5cb9-4280-8f3f-d6125a08e3e8 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:46.760267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04748eef-5cb9-4280-8f3f-d6125a08e3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:46.760549 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04748eef-5cb9-4280-8f3f-d6125a08e3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:46.778467 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-04748eef-5cb9-4280-8f3f-d6125a08e3e8 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:46.779584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-04748eef-5cb9-4280-8f3f-d6125a08e3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:46.846626 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 154/322] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:46.881105 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fd7a77-b72c-4d2f-8ef3-69e66c0ca910 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:46.970803 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:46.990988 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fd7a77-b72c-4d2f-8ef3-69e66c0ca910 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:47.031275 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43afe8c7-0920-4699-879e-34418b12c990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:47.031844 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43afe8c7-0920-4699-879e-34418b12c990 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:47.032142 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43afe8c7-0920-4699-879e-34418b12c990 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:47.032411 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43afe8c7-0920-4699-879e-34418b12c990 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:47.033483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43afe8c7-0920-4699-879e-34418b12c990 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:47.034974 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43afe8c7-0920-4699-879e-34418b12c990 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:47.040101 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 169/323] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:46:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:47.042579 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04748eef-5cb9-4280-8f3f-d6125a08e3e8 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:47.077340 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:47.100064 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e621a14-bca3-4c26-805a-cc7c0cf4ea94 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:47.100381 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e621a14-bca3-4c26-805a-cc7c0cf4ea94 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:47.100658 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e621a14-bca3-4c26-805a-cc7c0cf4ea94 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:47.102521 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 170/324] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:46:47.107273 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0ce0d-6afa-444a-98da-ccafedf5a99e None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:47.107548 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0ce0d-6afa-444a-98da-ccafedf5a99e None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:47.107816 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0ce0d-6afa-444a-98da-ccafedf5a99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:47.126439 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-1ba0ce0d-6afa-444a-98da-ccafedf5a99e None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:47.127620 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-1ba0ce0d-6afa-444a-98da-ccafedf5a99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:47.146972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04748eef-5cb9-4280-8f3f-d6125a08e3e8 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:47.180592 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d3e33-7e04-4f4c-8f90-f2c432eaeb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:47.181196 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d3e33-7e04-4f4c-8f90-f2c432eaeb37 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:47.181475 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d3e33-7e04-4f4c-8f90-f2c432eaeb37 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:47.181747 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d3e33-7e04-4f4c-8f90-f2c432eaeb37 None admin] PATH_INFO: `/v3/groups` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:47.182379 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8d3e33-7e04-4f4c-8f90-f2c432eaeb37 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:47.183817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8d3e33-7e04-4f4c-8f90-f2c432eaeb37 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:47.193789 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 171/325] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:46:47.194701 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba0ce0d-6afa-444a-98da-ccafedf5a99e None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:47.275158 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:47.287473 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 155/326] 10.222.0.48 () {60 vars in 1134 bytes} [Fri Sep 9 20:46:47 2022] POST /identity/v3/groups => generated 226 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:46:47.292117 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba0ce0d-6afa-444a-98da-ccafedf5a99e None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:47.334171 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca8fa4-ddf3-4088-b38e-fea8837ba71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:47.334814 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca8fa4-ddf3-4088-b38e-fea8837ba71d None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:47.335087 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca8fa4-ddf3-4088-b38e-fea8837ba71d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:47.335428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca8fa4-ddf3-4088-b38e-fea8837ba71d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:47.348790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca8fa4-ddf3-4088-b38e-fea8837ba71d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:47.354355 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-18ca8fa4-ddf3-4088-b38e-fea8837ba71d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:47.354355 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:47.354355 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:47.354355 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:47.354355 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:47.354355 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:47.354355 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:47.354355 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:47.354355 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:47.354355 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:47.354355 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:47.354355 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:47.354355 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:47.354355 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:47.354355 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:47.354355 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:47.355282 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:47.355282 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:47.355282 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:47.355282 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:47.355282 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:47.355282 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:47.355282 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:47.355282 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:47.355282 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:47.355282 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:47.355282 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:47.355282 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:47.355282 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:47.355282 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:47.355282 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:47.355282 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:47.356113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:47.356113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:47.356113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:47.356113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:47.356113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:47.356113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:47.356113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:47.356113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:47.356113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:47.356113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:47.356113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:47.356113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:46:47.356113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 172/327] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:47.363041 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d3e33-7e04-4f4c-8f90-f2c432eaeb37 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:47.464780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca8fa4-ddf3-4088-b38e-fea8837ba71d None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:47.467803 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:47.475099 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d3e33-7e04-4f4c-8f90-f2c432eaeb37 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:47.514825 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621263d8-dc5a-4652-9bb8-2dd678a9c602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:47.515547 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621263d8-dc5a-4652-9bb8-2dd678a9c602 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:47.515829 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621263d8-dc5a-4652-9bb8-2dd678a9c602 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:47.516176 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621263d8-dc5a-4652-9bb8-2dd678a9c602 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:47.516850 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621263d8-dc5a-4652-9bb8-2dd678a9c602 None admin] RBAC: Adding query filter params (name=member) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:47.519634 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621263d8-dc5a-4652-9bb8-2dd678a9c602 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:47.519634 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621263d8-dc5a-4652-9bb8-2dd678a9c602 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:47.527856 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 156/328] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Sep 9 20:46:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:47.533656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621263d8-dc5a-4652-9bb8-2dd678a9c602 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:47.608151 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca8fa4-ddf3-4088-b38e-fea8837ba71d None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:47.651584 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde693db-2ab9-48eb-ac8d-67d5a944dc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:47.652285 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde693db-2ab9-48eb-ac8d-67d5a944dc04 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:47.652571 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde693db-2ab9-48eb-ac8d-67d5a944dc04 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:47.653119 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde693db-2ab9-48eb-ac8d-67d5a944dc04 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:47.658481 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621263d8-dc5a-4652-9bb8-2dd678a9c602 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:47.667961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde693db-2ab9-48eb-ac8d-67d5a944dc04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:47.674621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-cde693db-2ab9-48eb-ac8d-67d5a944dc04 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:46:47.674621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:47.674621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:47.674621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:47.674621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:47.674621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:47.674621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:47.674621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:47.674621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:47.674621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:47.674621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:47.674621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:47.674621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:47.674621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:47.674621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:47.674621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:47.675523 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:47.675523 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:47.675523 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:47.675523 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:47.675523 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:47.675523 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:47.675523 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:47.675523 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:47.675523 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:47.675523 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:47.675523 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:47.675523 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:47.675523 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:47.675523 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:47.675523 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:47.675523 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:47.676372 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:47.676372 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:47.676372 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:47.676372 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:47.676372 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:47.676372 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:47.676372 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:47.676372 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:47.676372 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:47.676372 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:47.676372 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:46:47.676372 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:46:47.676372 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 173/329] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:47.681364 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde693db-2ab9-48eb-ac8d-67d5a944dc04 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:47.686826 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:47.699610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21ffc6-d800-4acf-85e8-4a16185d4119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:47.703365 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21ffc6-d800-4acf-85e8-4a16185d4119 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:47.703365 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21ffc6-d800-4acf-85e8-4a16185d4119 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:47.703365 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21ffc6-d800-4acf-85e8-4a16185d4119 None admin] PATH_INFO: `/v3/users/4d397df6246d4faebcfe0da3225afe0f` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:47.703365 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce21ffc6-d800-4acf-85e8-4a16185d4119 None admin] RBAC: Authorizing `identity:get_user(user_id=4d397df6246d4faebcfe0da3225afe0f)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:47.703624 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce21ffc6-d800-4acf-85e8-4a16185d4119 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:47.706423 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 157/330] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:47 2022] GET /identity/v3/users/4d397df6246d4faebcfe0da3225afe0f => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:46:47.712666 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21ffc6-d800-4acf-85e8-4a16185d4119 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:47.744845 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:47.804903 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde693db-2ab9-48eb-ac8d-67d5a944dc04 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:47.845155 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21ffc6-d800-4acf-85e8-4a16185d4119 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:47.853468 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e1ffd-adb0-4bb9-9c80-d8154a0712b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:47.854625 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e1ffd-adb0-4bb9-9c80-d8154a0712b9 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:47.854912 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e1ffd-adb0-4bb9-9c80-d8154a0712b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:47.855191 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e1ffd-adb0-4bb9-9c80-d8154a0712b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:47.857314 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e1ffd-adb0-4bb9-9c80-d8154a0712b9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:47.857640 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e1ffd-adb0-4bb9-9c80-d8154a0712b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:47.859395 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e1ffd-adb0-4bb9-9c80-d8154a0712b9 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:47.876171 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 174/331] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Sep 9 20:46:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:46:47.879482 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e1ffd-adb0-4bb9-9c80-d8154a0712b9 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:47.897388 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ecb05-9f4d-4679-925e-45103896cd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:47.898010 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ecb05-9f4d-4679-925e-45103896cd27 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:47.898389 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ecb05-9f4d-4679-925e-45103896cd27 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:47.898711 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ecb05-9f4d-4679-925e-45103896cd27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:47.899513 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05ecb05-9f4d-4679-925e-45103896cd27 None admin] RBAC: Adding query filter params (role.id=c59d891d042e48bab625f7b3666ef536, user.id=4d397df6246d4faebcfe0da3225afe0f, scope.system=all) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:47.899831 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05ecb05-9f4d-4679-925e-45103896cd27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:47.902583 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05ecb05-9f4d-4679-925e-45103896cd27 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:47.916215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 158/332] 10.222.0.48 () {56 vars in 1308 bytes} [Fri Sep 9 20:46:47 2022] GET /identity/v3/role_assignments?user.id=4d397df6246d4faebcfe0da3225afe0f&scope.system=all&role.id=c59d891d042e48bab625f7b3666ef536 => generated 510 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:46:47.974510 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:48.018255 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e1ffd-adb0-4bb9-9c80-d8154a0712b9 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:48.058175 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e2f70-be26-4a04-93ad-eaadba53368e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:48.058891 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e2f70-be26-4a04-93ad-eaadba53368e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:48.059178 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e2f70-be26-4a04-93ad-eaadba53368e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:48.059456 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e2f70-be26-4a04-93ad-eaadba53368e None admin] PATH_INFO: `/v3/users/b9270c31ba524eea9bb7ca9ae2c00a5e` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:48.060935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e2f70-be26-4a04-93ad-eaadba53368e None admin] RBAC: Authorizing `identity:get_user(user_id=b9270c31ba524eea9bb7ca9ae2c00a5e)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:48.062384 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e2f70-be26-4a04-93ad-eaadba53368e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:48.066713 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 175/333] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:47 2022] GET /identity/v3/users/b9270c31ba524eea9bb7ca9ae2c00a5e => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:48.070348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ecb05-9f4d-4679-925e-45103896cd27 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:48.176477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:48.197973 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ecb05-9f4d-4679-925e-45103896cd27 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:48.237609 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33c4d6-2585-4523-b6ff-c94ef24a2272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:48.238289 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33c4d6-2585-4523-b6ff-c94ef24a2272 None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:48.238642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33c4d6-2585-4523-b6ff-c94ef24a2272 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:48.238928 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33c4d6-2585-4523-b6ff-c94ef24a2272 None admin] PATH_INFO: `/v3/system/users/b9270c31ba524eea9bb7ca9ae2c00a5e/roles/e01870d6deef4f1ebbc7470c342046a6` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:48.240456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be33c4d6-2585-4523-b6ff-c94ef24a2272 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b9270c31ba524eea9bb7ca9ae2c00a5e, role_id=e01870d6deef4f1ebbc7470c342046a6)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:48.241933 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be33c4d6-2585-4523-b6ff-c94ef24a2272 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:48.303382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 159/334] 10.222.0.48 () {58 vars in 1334 bytes} [Fri Sep 9 20:46:48 2022] PUT /identity/v3/system/users/b9270c31ba524eea9bb7ca9ae2c00a5e/roles/e01870d6deef4f1ebbc7470c342046a6 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:46:49.990096 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02760e7c-6b6b-4af4-9b94-823c7f91bad1 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:49.990625 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02760e7c-6b6b-4af4-9b94-823c7f91bad1 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:49.990625 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02760e7c-6b6b-4af4-9b94-823c7f91bad1 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:49.992278 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 176/335] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:49.998321 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619abd4-d7ed-4e20-803b-94625a2eebe2 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:49.998827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619abd4-d7ed-4e20-803b-94625a2eebe2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:49.998901 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619abd4-d7ed-4e20-803b-94625a2eebe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:50.025994 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-7619abd4-d7ed-4e20-803b-94625a2eebe2 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:50.027680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7619abd4-d7ed-4e20-803b-94625a2eebe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:50.111254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 160/336] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:50.283425 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e212a-c853-48bb-92b8-899adf2c8a9f None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:50.283607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e212a-c853-48bb-92b8-899adf2c8a9f None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:50.283881 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e212a-c853-48bb-92b8-899adf2c8a9f None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:50.285404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 177/337] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:50.294529 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd16ff6-61c0-4a3c-8aa4-0e60263cb932 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:50.294824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd16ff6-61c0-4a3c-8aa4-0e60263cb932 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:50.295105 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd16ff6-61c0-4a3c-8aa4-0e60263cb932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:50.318194 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-8bd16ff6-61c0-4a3c-8aa4-0e60263cb932 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:50.319457 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8bd16ff6-61c0-4a3c-8aa4-0e60263cb932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:50.412216 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 161/338] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:50.693693 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e212a-c853-48bb-92b8-899adf2c8a9f None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:50.716993 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd16ff6-61c0-4a3c-8aa4-0e60263cb932 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:50.730042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:50.884061 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e212a-c853-48bb-92b8-899adf2c8a9f None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:50.887542 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd16ff6-61c0-4a3c-8aa4-0e60263cb932 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:50.926051 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2eecc3-0a02-48a6-a851-1c363785a7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:50.926702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2eecc3-0a02-48a6-a851-1c363785a7f8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:50.926982 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2eecc3-0a02-48a6-a851-1c363785a7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:50.927261 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2eecc3-0a02-48a6-a851-1c363785a7f8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:50.946088 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2eecc3-0a02-48a6-a851-1c363785a7f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:50.951357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-1d2eecc3-0a02-48a6-a851-1c363785a7f8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:46:50.951357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:50.951357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:50.951357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:50.951357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:50.951357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:50.951357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:50.951357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:50.951357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:50.951357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:50.951357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:50.951357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:50.951357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:50.951357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:50.951357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:50.951357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:50.952455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:50.952455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:50.952455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:50.952455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:50.952455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:50.952455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:50.952455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:50.952455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:50.952455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:50.952455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:50.952455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:50.952455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:50.952455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:50.952455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:50.952455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:50.952455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:50.953444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:50.953444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:50.953444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:50.953444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:50.953444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:50.953444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:50.953444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:50.953444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:50.953444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:50.953444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:50.953444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:46:50.953444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:46:50.955724 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 162/339] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Sep 9 20:46:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:50.957746 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af459fd9-c8e6-463c-8855-15721df657c2 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:50.958186 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af459fd9-c8e6-463c-8855-15721df657c2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:50.958646 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af459fd9-c8e6-463c-8855-15721df657c2 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:50.960364 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 163/340] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:50.965964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af459fd9-c8e6-463c-8855-15721df657c2 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:50.997361 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7b695-b4fa-4b20-a5eb-2973197d78d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:50.998074 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7b695-b4fa-4b20-a5eb-2973197d78d7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:50.998354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7b695-b4fa-4b20-a5eb-2973197d78d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:50.998624 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7b695-b4fa-4b20-a5eb-2973197d78d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:51.001661 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:51.012585 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c7b695-b4fa-4b20-a5eb-2973197d78d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:51.022187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-f2c7b695-b4fa-4b20-a5eb-2973197d78d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:51.022187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:51.022187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:51.022187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:51.022187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:51.022187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:51.022187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:51.022187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:51.022187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:51.022187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:51.022187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:51.022187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:51.022187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:51.022187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:51.022187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:51.022187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:51.023356 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:51.023356 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:51.023356 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:51.023356 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:51.023356 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:51.023356 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:51.023356 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:51.023356 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:51.023356 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:51.023356 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:51.023356 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:51.023356 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:51.023356 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:51.023356 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:51.023356 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:51.023356 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:51.024770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:51.024770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:51.024770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:51.024770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:51.024770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:51.024770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:51.024770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:51.024770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:51.024770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:51.024770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:51.024770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:51.024770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:46:51.024770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 178/341] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:51.024770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd30cb-634c-4d27-8545-ddd7b9cacc00 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:51.025510 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd30cb-634c-4d27-8545-ddd7b9cacc00 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:51.025510 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd30cb-634c-4d27-8545-ddd7b9cacc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:51.045617 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-65fd30cb-634c-4d27-8545-ddd7b9cacc00 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:51.046704 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-65fd30cb-634c-4d27-8545-ddd7b9cacc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:51.076099 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:51.118203 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 179/342] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:51.118800 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd30cb-634c-4d27-8545-ddd7b9cacc00 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:51.133903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af459fd9-c8e6-463c-8855-15721df657c2 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:51.187356 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe7986-56f0-4d49-b717-691fd698ebfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:51.188246 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe7986-56f0-4d49-b717-691fd698ebfb None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:51.188729 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe7986-56f0-4d49-b717-691fd698ebfb None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:51.189177 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe7986-56f0-4d49-b717-691fd698ebfb None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:51.190102 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe7986-56f0-4d49-b717-691fd698ebfb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:51.190556 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe7986-56f0-4d49-b717-691fd698ebfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:51.192189 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe7986-56f0-4d49-b717-691fd698ebfb None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:51.205421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 164/343] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:46:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:51.210642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe7986-56f0-4d49-b717-691fd698ebfb None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:51.229063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd30cb-634c-4d27-8545-ddd7b9cacc00 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:51.255157 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:51.267390 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a008a0e9-f75f-464b-b1ad-0efdefba69da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:51.268010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a008a0e9-f75f-464b-b1ad-0efdefba69da None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:51.272339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a008a0e9-f75f-464b-b1ad-0efdefba69da None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:51.272339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a008a0e9-f75f-464b-b1ad-0efdefba69da None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:51.272339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a008a0e9-f75f-464b-b1ad-0efdefba69da None admin] RBAC: Adding query filter params (name=member) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:51.272339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a008a0e9-f75f-464b-b1ad-0efdefba69da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:51.273203 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a008a0e9-f75f-464b-b1ad-0efdefba69da None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:51.287587 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 180/344] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Sep 9 20:46:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:51.294733 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a008a0e9-f75f-464b-b1ad-0efdefba69da None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:51.393557 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe7986-56f0-4d49-b717-691fd698ebfb None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:51.434908 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:51.443836 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1e23f-39c1-495a-a94f-6922795496f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:51.449612 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1e23f-39c1-495a-a94f-6922795496f0 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:51.449612 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1e23f-39c1-495a-a94f-6922795496f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:51.449612 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1e23f-39c1-495a-a94f-6922795496f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:51.509237 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a008a0e9-f75f-464b-b1ad-0efdefba69da None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:51.551838 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca5157e-1bbc-4bc8-ba1a-cbc027b87043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:51.552480 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5157e-1bbc-4bc8-ba1a-cbc027b87043 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:51.552759 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5157e-1bbc-4bc8-ba1a-cbc027b87043 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:51.553030 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5157e-1bbc-4bc8-ba1a-cbc027b87043 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:51.555572 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1e23f-39c1-495a-a94f-6922795496f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:51.557321 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1e23f-39c1-495a-a94f-6922795496f0 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:51.611931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:51.626212 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca5157e-1bbc-4bc8-ba1a-cbc027b87043 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:51.627709 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca5157e-1bbc-4bc8-ba1a-cbc027b87043 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:51.667567 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 165/345] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:46:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:46:51.669033 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12513892-fb67-4afa-8aa0-0934d8c07ce9 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:51.669331 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12513892-fb67-4afa-8aa0-0934d8c07ce9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:51.669611 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12513892-fb67-4afa-8aa0-0934d8c07ce9 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:51.672905 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 166/346] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:51.673673 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12513892-fb67-4afa-8aa0-0934d8c07ce9 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:51.711489 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:51.741818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 181/347] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:46:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 20:46:51.743224 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca5157e-1bbc-4bc8-ba1a-cbc027b87043 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:51.796139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12513892-fb67-4afa-8aa0-0934d8c07ce9 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:51.834229 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925c614-a30a-456e-a6e6-6d4b0c436457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:51.834888 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925c614-a30a-456e-a6e6-6d4b0c436457 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:51.835167 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925c614-a30a-456e-a6e6-6d4b0c436457 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:51.835439 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925c614-a30a-456e-a6e6-6d4b0c436457 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:51.837900 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:51.848063 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925c614-a30a-456e-a6e6-6d4b0c436457 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:51.854231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-6925c614-a30a-456e-a6e6-6d4b0c436457 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:51.854231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:51.854231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:51.854231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:51.854231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:51.854231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:51.854231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:51.854231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:51.854231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:51.854231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:51.854231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:51.854231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:51.854231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:51.854231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:51.854231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:51.854231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:51.855219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:51.855219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:51.855219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:51.855219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:51.855219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:51.855219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:51.855219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:51.855219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:51.855219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:51.855219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:51.855219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:51.855219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:51.855219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:51.855219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:51.855219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:51.855219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:51.856472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:51.856472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:51.856472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:51.856472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:51.856472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:51.856472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:51.856472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:51.856472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:51.856472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:51.856472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:51.856472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:51.856472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:46:51.857102 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca5157e-1bbc-4bc8-ba1a-cbc027b87043 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:51.857173 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 167/348] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:51.857520 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63874c1b-3a63-4224-8c5e-37e674775cee None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:51.857706 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63874c1b-3a63-4224-8c5e-37e674775cee None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:51.857980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63874c1b-3a63-4224-8c5e-37e674775cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:51.885008 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-63874c1b-3a63-4224-8c5e-37e674775cee None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:51.886210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-63874c1b-3a63-4224-8c5e-37e674775cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:51.895571 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e61540-3ddf-4eeb-a182-f277107e3b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:51.896238 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e61540-3ddf-4eeb-a182-f277107e3b61 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:51.896535 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e61540-3ddf-4eeb-a182-f277107e3b61 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:51.896777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e61540-3ddf-4eeb-a182-f277107e3b61 None admin] PATH_INFO: `/v3/projects/1464b3dd76734fb593f3ed3582957e86` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:51.897904 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e61540-3ddf-4eeb-a182-f277107e3b61 None admin] RBAC: Authorizing `identity:get_project(project_id=1464b3dd76734fb593f3ed3582957e86)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:51.899382 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e61540-3ddf-4eeb-a182-f277107e3b61 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:51.901035 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 182/349] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:51 2022] GET /identity/v3/projects/1464b3dd76734fb593f3ed3582957e86 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:51.901796 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e61540-3ddf-4eeb-a182-f277107e3b61 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:51.964206 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 168/350] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:51.965205 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63874c1b-3a63-4224-8c5e-37e674775cee None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:52.012464 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:52.028183 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e61540-3ddf-4eeb-a182-f277107e3b61 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:52.051711 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd967c83-0943-459f-9fbe-7a7e485f1f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:52.052319 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd967c83-0943-459f-9fbe-7a7e485f1f25 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:52.052600 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd967c83-0943-459f-9fbe-7a7e485f1f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:52.052870 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd967c83-0943-459f-9fbe-7a7e485f1f25 None admin] PATH_INFO: `/v3/projects/1464b3dd76734fb593f3ed3582957e86` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:52.053999 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd967c83-0943-459f-9fbe-7a7e485f1f25 None admin] RBAC: Authorizing `identity:get_project(project_id=1464b3dd76734fb593f3ed3582957e86)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:52.055996 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd967c83-0943-459f-9fbe-7a7e485f1f25 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:52.058690 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 183/351] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:51 2022] GET /identity/v3/projects/1464b3dd76734fb593f3ed3582957e86 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:52.059476 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd967c83-0943-459f-9fbe-7a7e485f1f25 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:52.113220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63874c1b-3a63-4224-8c5e-37e674775cee None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:52.116462 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:52.155148 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f990b7a-1c43-4139-88e7-adbf939c8300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:52.155772 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f990b7a-1c43-4139-88e7-adbf939c8300 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:52.156049 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f990b7a-1c43-4139-88e7-adbf939c8300 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:52.156364 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f990b7a-1c43-4139-88e7-adbf939c8300 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:52.157040 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f990b7a-1c43-4139-88e7-adbf939c8300 None admin] RBAC: Adding query filter params (name=member) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:52.157346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f990b7a-1c43-4139-88e7-adbf939c8300 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:52.158814 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f990b7a-1c43-4139-88e7-adbf939c8300 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:52.168864 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 169/352] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Sep 9 20:46:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:52.170084 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9b5f1-49d0-4662-abff-cf16f9e69d91 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:52.170368 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9b5f1-49d0-4662-abff-cf16f9e69d91 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:52.170639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9b5f1-49d0-4662-abff-cf16f9e69d91 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:52.172097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 170/353] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:52.172828 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9b5f1-49d0-4662-abff-cf16f9e69d91 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:52.188943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd967c83-0943-459f-9fbe-7a7e485f1f25 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:52.240597 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b4d33-9ecd-41ae-b917-caebf0e00408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:52.241247 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b4d33-9ecd-41ae-b917-caebf0e00408 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:52.241525 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b4d33-9ecd-41ae-b917-caebf0e00408 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:52.241798 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b4d33-9ecd-41ae-b917-caebf0e00408 None admin] PATH_INFO: `/v3/groups/2a4806378be54b8d9473e70d7eff9171` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:52.251787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17b4d33-9ecd-41ae-b917-caebf0e00408 None admin] RBAC: Authorizing `identity:get_group(group_id=2a4806378be54b8d9473e70d7eff9171)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:52.253275 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17b4d33-9ecd-41ae-b917-caebf0e00408 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:52.255207 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 184/354] 10.222.0.48 () {56 vars in 1180 bytes} [Fri Sep 9 20:46:52 2022] GET /identity/v3/groups/2a4806378be54b8d9473e70d7eff9171 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:52.256106 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b4d33-9ecd-41ae-b917-caebf0e00408 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:52.286116 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:52.313773 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9b5f1-49d0-4662-abff-cf16f9e69d91 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:52.361812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f09ae-3921-4881-86f8-606687314a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:52.362468 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f09ae-3921-4881-86f8-606687314a71 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:52.362762 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f09ae-3921-4881-86f8-606687314a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:52.363006 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f09ae-3921-4881-86f8-606687314a71 None admin] PATH_INFO: `/v3/groups/2a4806378be54b8d9473e70d7eff9171` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:52.363946 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:52.364174 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f09ae-3921-4881-86f8-606687314a71 None admin] RBAC: Authorizing `identity:get_group(group_id=2a4806378be54b8d9473e70d7eff9171)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:52.366711 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f09ae-3921-4881-86f8-606687314a71 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:52.369286 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 171/355] 10.222.0.48 () {56 vars in 1180 bytes} [Fri Sep 9 20:46:52 2022] GET /identity/v3/groups/2a4806378be54b8d9473e70d7eff9171 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:52.370818 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d781008-d645-486a-8969-6b6d77b9618c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:52.370818 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d781008-d645-486a-8969-6b6d77b9618c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:52.370989 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d781008-d645-486a-8969-6b6d77b9618c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:52.386361 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b4d33-9ecd-41ae-b917-caebf0e00408 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:52.395317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-4d781008-d645-486a-8969-6b6d77b9618c None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:52.396618 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4d781008-d645-486a-8969-6b6d77b9618c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:52.443124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc066d4e-c30f-4683-9809-7ace6c52b17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:52.443745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc066d4e-c30f-4683-9809-7ace6c52b17b None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:52.444025 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc066d4e-c30f-4683-9809-7ace6c52b17b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:52.445093 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc066d4e-c30f-4683-9809-7ace6c52b17b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:52.485115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 172/356] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:52.486261 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d781008-d645-486a-8969-6b6d77b9618c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:52.527066 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc066d4e-c30f-4683-9809-7ace6c52b17b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:52.528759 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc066d4e-c30f-4683-9809-7ace6c52b17b None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:52.578460 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:52.607761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d781008-d645-486a-8969-6b6d77b9618c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:52.637707 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:52.645761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a183da-92ef-4380-b9af-b6c9c6841c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:52.646584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a183da-92ef-4380-b9af-b6c9c6841c43 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:52.646879 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a183da-92ef-4380-b9af-b6c9c6841c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:52.647127 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a183da-92ef-4380-b9af-b6c9c6841c43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:52.647863 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a183da-92ef-4380-b9af-b6c9c6841c43 None admin] RBAC: Adding query filter params (group.id=2a4806378be54b8d9473e70d7eff9171, role.id=13072ad4b81649459f59b8137db492ec, scope.project.id=1464b3dd76734fb593f3ed3582957e86) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:52.648216 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a183da-92ef-4380-b9af-b6c9c6841c43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:52.650196 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a183da-92ef-4380-b9af-b6c9c6841c43 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:52.656568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 185/357] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:46:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:46:52.656709 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 173/358] 10.222.0.48 () {56 vars in 1376 bytes} [Fri Sep 9 20:46:52 2022] GET /identity/v3/role_assignments?group.id=2a4806378be54b8d9473e70d7eff9171&scope.project.id=1464b3dd76734fb593f3ed3582957e86&role.id=13072ad4b81649459f59b8137db492ec => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:52.657232 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a183da-92ef-4380-b9af-b6c9c6841c43 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:52.664739 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc066d4e-c30f-4683-9809-7ace6c52b17b None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:52.790125 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a183da-92ef-4380-b9af-b6c9c6841c43 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:52.833361 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b18d2-29c8-4bd1-bbc5-4225b3807449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:52.833901 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b18d2-29c8-4bd1-bbc5-4225b3807449 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:52.834178 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b18d2-29c8-4bd1-bbc5-4225b3807449 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:52.834442 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b18d2-29c8-4bd1-bbc5-4225b3807449 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:52.835195 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b18d2-29c8-4bd1-bbc5-4225b3807449 None admin] RBAC: Adding query filter params (group.id=2a4806378be54b8d9473e70d7eff9171, role.id=c59d891d042e48bab625f7b3666ef536, scope.project.id=1464b3dd76734fb593f3ed3582957e86) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:52.835501 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b18d2-29c8-4bd1-bbc5-4225b3807449 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:52.837597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b18d2-29c8-4bd1-bbc5-4225b3807449 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:52.843889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 174/359] 10.222.0.48 () {56 vars in 1376 bytes} [Fri Sep 9 20:46:52 2022] GET /identity/v3/role_assignments?group.id=2a4806378be54b8d9473e70d7eff9171&scope.project.id=1464b3dd76734fb593f3ed3582957e86&role.id=c59d891d042e48bab625f7b3666ef536 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:52.845287 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca94a6-0b0f-4069-a21c-4cdae6a28383 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:52.845596 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc066d4e-c30f-4683-9809-7ace6c52b17b None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:52.846576 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca94a6-0b0f-4069-a21c-4cdae6a28383 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:52.846844 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca94a6-0b0f-4069-a21c-4cdae6a28383 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:52.848398 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 175/360] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:52.853325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca94a6-0b0f-4069-a21c-4cdae6a28383 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:52.866796 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:52.889514 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf351b0-cb64-41ea-9678-58b657bf2ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:52.890267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf351b0-cb64-41ea-9678-58b657bf2ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:52.890582 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf351b0-cb64-41ea-9678-58b657bf2ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:52.890803 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf351b0-cb64-41ea-9678-58b657bf2ae9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:52.906095 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf351b0-cb64-41ea-9678-58b657bf2ae9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:52.913127 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-baf351b0-cb64-41ea-9678-58b657bf2ae9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:46:52.913127 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:52.913127 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:52.913127 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:52.913127 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:52.913127 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:52.913127 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:52.913127 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:52.913127 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:52.913127 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:52.913127 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:52.913127 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:52.913127 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:52.913127 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:52.913127 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:52.913127 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:52.914143 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:52.914143 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:52.914143 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:52.914143 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:52.914143 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:52.914143 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:52.914143 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:52.914143 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:52.914143 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:52.914143 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:52.914143 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:52.914143 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:52.914143 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:52.914143 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:52.914143 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:52.914143 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:52.915420 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:52.915420 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:52.915420 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:52.915420 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:52.915420 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:52.915420 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:52.915420 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:52.915420 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:52.915420 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:52.915420 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:52.915420 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 20:46:52.915420 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:46:52.915420 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 186/361] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:52.916104 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf351b0-cb64-41ea-9678-58b657bf2ae9 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:52.983609 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:52.997683 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca94a6-0b0f-4069-a21c-4cdae6a28383 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:53.046543 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187eaa7-d462-414a-b911-b7ca47774e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:53.047219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187eaa7-d462-414a-b911-b7ca47774e75 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:53.047503 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187eaa7-d462-414a-b911-b7ca47774e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:53.047783 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187eaa7-d462-414a-b911-b7ca47774e75 None admin] PATH_INFO: `/v3/projects/e13bca418ede487ab7d1230e28403173` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:53.049755 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187eaa7-d462-414a-b911-b7ca47774e75 None admin] RBAC: Authorizing `identity:get_project(project_id=e13bca418ede487ab7d1230e28403173)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:53.051622 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187eaa7-d462-414a-b911-b7ca47774e75 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:53.054231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 176/362] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:52 2022] GET /identity/v3/projects/e13bca418ede487ab7d1230e28403173 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:53.061476 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee66550-b700-435e-887c-a90e2ed8636c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:53.061476 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee66550-b700-435e-887c-a90e2ed8636c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:53.061476 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee66550-b700-435e-887c-a90e2ed8636c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:53.064122 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf351b0-cb64-41ea-9678-58b657bf2ae9 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:53.080280 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-8ee66550-b700-435e-887c-a90e2ed8636c None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:53.081543 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8ee66550-b700-435e-887c-a90e2ed8636c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:53.121683 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218fe3f6-9164-479a-a152-10ceb3361300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:53.122691 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fe3f6-9164-479a-a152-10ceb3361300 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:53.123195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fe3f6-9164-479a-a152-10ceb3361300 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:53.123687 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fe3f6-9164-479a-a152-10ceb3361300 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:53.139721 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218fe3f6-9164-479a-a152-10ceb3361300 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:53.145517 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-218fe3f6-9164-479a-a152-10ceb3361300 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:46:53.145517 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:53.145517 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:53.145517 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:53.145517 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:53.145517 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:53.145517 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:53.145517 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:53.145517 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:53.145517 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:53.145517 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:53.145517 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:53.145517 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:53.145517 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:53.145517 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:53.145517 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:53.146426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:53.146426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:53.146426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:53.146426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:53.146426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:53.146426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:53.146426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:53.146426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:53.146426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:53.146426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:53.146426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:53.146426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:53.146426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:53.146426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:53.146426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:53.146426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:53.147232 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:53.147232 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:53.147232 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:53.147232 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:53.147232 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:53.147232 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:53.147232 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:53.147232 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:53.147232 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:53.147232 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:53.147232 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:46:53.147232 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:46:53.150163 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 187/363] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Sep 9 20:46:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:53.156277 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218fe3f6-9164-479a-a152-10ceb3361300 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:53.180148 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 177/364] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:53.181122 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee66550-b700-435e-887c-a90e2ed8636c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:53.184352 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:53.283614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218fe3f6-9164-479a-a152-10ceb3361300 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:53.292584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee66550-b700-435e-887c-a90e2ed8636c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:53.292759 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:53.334281 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15469c-14e5-46fa-9953-adc8bb59d2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:53.334899 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15469c-14e5-46fa-9953-adc8bb59d2ff None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:53.335188 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15469c-14e5-46fa-9953-adc8bb59d2ff None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:53.335459 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15469c-14e5-46fa-9953-adc8bb59d2ff None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:53.336194 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c15469c-14e5-46fa-9953-adc8bb59d2ff None admin] RBAC: Adding query filter params (name=reader) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:53.336500 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c15469c-14e5-46fa-9953-adc8bb59d2ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:53.342489 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c15469c-14e5-46fa-9953-adc8bb59d2ff None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:53.354793 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9da08-fd00-42dc-8ab5-94926cd2743c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:53.355443 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9da08-fd00-42dc-8ab5-94926cd2743c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:53.355718 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9da08-fd00-42dc-8ab5-94926cd2743c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:53.355991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9da08-fd00-42dc-8ab5-94926cd2743c None admin] PATH_INFO: `/v3/groups/2a4806378be54b8d9473e70d7eff9171` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:53.357148 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9da08-fd00-42dc-8ab5-94926cd2743c None admin] RBAC: Authorizing `identity:get_group(group_id=2a4806378be54b8d9473e70d7eff9171)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:53.359925 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 188/365] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Sep 9 20:46:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:53.360680 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15469c-14e5-46fa-9953-adc8bb59d2ff None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:53.361818 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9da08-fd00-42dc-8ab5-94926cd2743c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:53.368277 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 178/366] 10.222.0.48 () {56 vars in 1180 bytes} [Fri Sep 9 20:46:53 2022] GET /identity/v3/groups/2a4806378be54b8d9473e70d7eff9171 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:53.369156 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9da08-fd00-42dc-8ab5-94926cd2743c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:53.513171 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:53.571030 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9da08-fd00-42dc-8ab5-94926cd2743c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:53.575780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15469c-14e5-46fa-9953-adc8bb59d2ff None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:53.613720 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c9d6b-d95f-42c0-9213-157defbd5c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:53.615469 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c9d6b-d95f-42c0-9213-157defbd5c23 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:53.615984 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c9d6b-d95f-42c0-9213-157defbd5c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:53.616500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c9d6b-d95f-42c0-9213-157defbd5c23 None admin] PATH_INFO: `/v3/users/b9270c31ba524eea9bb7ca9ae2c00a5e` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:53.620726 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c9d6b-d95f-42c0-9213-157defbd5c23 None admin] RBAC: Authorizing `identity:get_user(user_id=b9270c31ba524eea9bb7ca9ae2c00a5e)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:53.622430 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c9d6b-d95f-42c0-9213-157defbd5c23 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:53.624672 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c6638-9890-411f-832d-3108fa75f206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:53.624803 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 179/367] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:46:53 2022] GET /identity/v3/users/b9270c31ba524eea9bb7ca9ae2c00a5e => generated 293 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:53.625290 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c6638-9890-411f-832d-3108fa75f206 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:53.625570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c6638-9890-411f-832d-3108fa75f206 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:53.625844 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c6638-9890-411f-832d-3108fa75f206 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:53.626444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c9d6b-d95f-42c0-9213-157defbd5c23 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:53.626548 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2c6638-9890-411f-832d-3108fa75f206 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:53.626806 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2c6638-9890-411f-832d-3108fa75f206 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:53.633129 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2c6638-9890-411f-832d-3108fa75f206 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:53.642909 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 189/368] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:46:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:53.643868 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c6638-9890-411f-832d-3108fa75f206 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:53.691613 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:53.760812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c9d6b-d95f-42c0-9213-157defbd5c23 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:53.768049 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c6638-9890-411f-832d-3108fa75f206 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:53.805580 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a91dbb-6b15-4a77-8cfd-a2ffd292aa6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:53.806251 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a91dbb-6b15-4a77-8cfd-a2ffd292aa6b None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:53.806526 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a91dbb-6b15-4a77-8cfd-a2ffd292aa6b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:53.806795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a91dbb-6b15-4a77-8cfd-a2ffd292aa6b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:53.814333 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd22ddc-be92-4e2f-8b16-9827e9d0ac56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:53.814959 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd22ddc-be92-4e2f-8b16-9827e9d0ac56 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:53.815346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd22ddc-be92-4e2f-8b16-9827e9d0ac56 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:53.815627 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd22ddc-be92-4e2f-8b16-9827e9d0ac56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:53.816537 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd22ddc-be92-4e2f-8b16-9827e9d0ac56 None admin] RBAC: Adding query filter params (group.id=2a4806378be54b8d9473e70d7eff9171, role.id=c59d891d042e48bab625f7b3666ef536, scope.project.id=e13bca418ede487ab7d1230e28403173) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:53.816852 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd22ddc-be92-4e2f-8b16-9827e9d0ac56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:53.818341 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd22ddc-be92-4e2f-8b16-9827e9d0ac56 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:53.819328 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a91dbb-6b15-4a77-8cfd-a2ffd292aa6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:53.824439 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-13a91dbb-6b15-4a77-8cfd-a2ffd292aa6b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:46:53.824439 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:53.824439 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:53.824439 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:53.824439 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:53.824439 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:53.824439 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:53.824439 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:53.824439 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:53.824439 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:53.824439 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:53.824439 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:53.824439 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:53.824439 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:53.824439 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:53.824439 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:53.825843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:53.825843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:53.825843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:53.825843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:53.825843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:53.825843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:53.825843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:53.825843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:53.825843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:53.825843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:53.825843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:53.825843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:53.825843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:53.825843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:53.825843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:53.825843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:53.827010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:53.827010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:53.827010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:53.827010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:53.827010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:53.827010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:53.827010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:53.827010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:53.827010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:53.827010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:53.827010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:46:53.827010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:46:53.827958 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 180/369] 10.222.0.48 () {56 vars in 1376 bytes} [Fri Sep 9 20:46:53 2022] GET /identity/v3/role_assignments?group.id=2a4806378be54b8d9473e70d7eff9171&scope.project.id=e13bca418ede487ab7d1230e28403173&role.id=c59d891d042e48bab625f7b3666ef536 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:53.827958 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd22ddc-be92-4e2f-8b16-9827e9d0ac56 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:53.828217 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 190/370] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:46:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:53.828217 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a91dbb-6b15-4a77-8cfd-a2ffd292aa6b None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:53.834588 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:53.938435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:53.949123 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a91dbb-6b15-4a77-8cfd-a2ffd292aa6b None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:53.953655 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd22ddc-be92-4e2f-8b16-9827e9d0ac56 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:53.993338 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6348fd-3446-4703-9101-6836d57cf19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:53.993674 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cdad50-6e9d-48f5-a72b-b8288458b8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:53.993955 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6348fd-3446-4703-9101-6836d57cf19c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:53.994124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cdad50-6e9d-48f5-a72b-b8288458b8df None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:53.994293 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6348fd-3446-4703-9101-6836d57cf19c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:53.994457 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cdad50-6e9d-48f5-a72b-b8288458b8df None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:53.994605 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6348fd-3446-4703-9101-6836d57cf19c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:53.994748 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cdad50-6e9d-48f5-a72b-b8288458b8df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:53.995314 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6348fd-3446-4703-9101-6836d57cf19c None admin] RBAC: Adding query filter params (role.id=e01870d6deef4f1ebbc7470c342046a6, user.id=b9270c31ba524eea9bb7ca9ae2c00a5e, scope.system=all) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:53.995633 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6348fd-3446-4703-9101-6836d57cf19c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:53.997107 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6348fd-3446-4703-9101-6836d57cf19c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:54.010444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 181/371] 10.222.0.48 () {56 vars in 1308 bytes} [Fri Sep 9 20:46:53 2022] GET /identity/v3/role_assignments?user.id=b9270c31ba524eea9bb7ca9ae2c00a5e&scope.system=all&role.id=e01870d6deef4f1ebbc7470c342046a6 => generated 510 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:54.011292 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6348fd-3446-4703-9101-6836d57cf19c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:54.081543 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:54.099716 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cdad50-6e9d-48f5-a72b-b8288458b8df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:54.102353 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cdad50-6e9d-48f5-a72b-b8288458b8df None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:54.128106 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6348fd-3446-4703-9101-6836d57cf19c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:54.214349 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:54.224802 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 191/372] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:46:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:46:54.231649 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cdad50-6e9d-48f5-a72b-b8288458b8df None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:54.237695 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24810dcb-b076-40a5-938a-392dc55e0a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:54.238380 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24810dcb-b076-40a5-938a-392dc55e0a70 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:54.238663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24810dcb-b076-40a5-938a-392dc55e0a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:54.238935 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24810dcb-b076-40a5-938a-392dc55e0a70 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:54.239623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24810dcb-b076-40a5-938a-392dc55e0a70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:54.239926 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24810dcb-b076-40a5-938a-392dc55e0a70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:54.241353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24810dcb-b076-40a5-938a-392dc55e0a70 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:54.263324 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 182/373] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:54.269138 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24810dcb-b076-40a5-938a-392dc55e0a70 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:54.386031 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cdad50-6e9d-48f5-a72b-b8288458b8df None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:54.448850 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24810dcb-b076-40a5-938a-392dc55e0a70 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:54.453720 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b41427-ef73-4529-81ac-5ab57cdb1e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:54.454395 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b41427-ef73-4529-81ac-5ab57cdb1e38 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:54.454655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b41427-ef73-4529-81ac-5ab57cdb1e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:54.454924 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b41427-ef73-4529-81ac-5ab57cdb1e38 None admin] PATH_INFO: `/v3/projects/e13bca418ede487ab7d1230e28403173` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:54.456107 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b41427-ef73-4529-81ac-5ab57cdb1e38 None admin] RBAC: Authorizing `identity:get_project(project_id=e13bca418ede487ab7d1230e28403173)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:54.457645 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b41427-ef73-4529-81ac-5ab57cdb1e38 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:54.459304 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 192/374] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:54 2022] GET /identity/v3/projects/e13bca418ede487ab7d1230e28403173 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:54.467374 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b41427-ef73-4529-81ac-5ab57cdb1e38 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:54.488364 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:54.509442 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5df178-5993-46aa-ad0c-5c0f2b10dc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:54.510061 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5df178-5993-46aa-ad0c-5c0f2b10dc80 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:54.510345 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5df178-5993-46aa-ad0c-5c0f2b10dc80 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:54.510619 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5df178-5993-46aa-ad0c-5c0f2b10dc80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:54.568436 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:54.642737 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5df178-5993-46aa-ad0c-5c0f2b10dc80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:54.644157 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5df178-5993-46aa-ad0c-5c0f2b10dc80 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:54.649573 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b41427-ef73-4529-81ac-5ab57cdb1e38 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:54.705272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1f3fe-74b9-482b-84c3-a25980edf735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:54.705910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1f3fe-74b9-482b-84c3-a25980edf735 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:54.706200 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1f3fe-74b9-482b-84c3-a25980edf735 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:54.706472 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1f3fe-74b9-482b-84c3-a25980edf735 None admin] PATH_INFO: `/v3/groups/2a4806378be54b8d9473e70d7eff9171` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:54.707711 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1f3fe-74b9-482b-84c3-a25980edf735 None admin] RBAC: Authorizing `identity:get_group(group_id=2a4806378be54b8d9473e70d7eff9171)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:54.709214 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1f3fe-74b9-482b-84c3-a25980edf735 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:54.710822 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 193/375] 10.222.0.48 () {56 vars in 1180 bytes} [Fri Sep 9 20:46:54 2022] GET /identity/v3/groups/2a4806378be54b8d9473e70d7eff9171 => generated 226 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:54.719743 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1f3fe-74b9-482b-84c3-a25980edf735 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:54.849629 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 183/376] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:46:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 20:46:54.855745 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5df178-5993-46aa-ad0c-5c0f2b10dc80 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:54.904448 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:54.922220 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1f3fe-74b9-482b-84c3-a25980edf735 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:54.963379 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:54.980105 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21220248-2cdc-490b-8f9c-cf49e419e137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:54.980105 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21220248-2cdc-490b-8f9c-cf49e419e137 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:54.980105 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21220248-2cdc-490b-8f9c-cf49e419e137 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:54.980105 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21220248-2cdc-490b-8f9c-cf49e419e137 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:54.982556 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21220248-2cdc-490b-8f9c-cf49e419e137 None admin] RBAC: Adding query filter params (group.id=2a4806378be54b8d9473e70d7eff9171, role.id=13072ad4b81649459f59b8137db492ec, scope.project.id=e13bca418ede487ab7d1230e28403173) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:54.982556 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21220248-2cdc-490b-8f9c-cf49e419e137 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:54.982556 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21220248-2cdc-490b-8f9c-cf49e419e137 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:54.996732 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 194/377] 10.222.0.48 () {56 vars in 1376 bytes} [Fri Sep 9 20:46:54 2022] GET /identity/v3/role_assignments?group.id=2a4806378be54b8d9473e70d7eff9171&scope.project.id=e13bca418ede487ab7d1230e28403173&role.id=13072ad4b81649459f59b8137db492ec => generated 260 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:55.052293 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5df178-5993-46aa-ad0c-5c0f2b10dc80 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:55.107138 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b86f7b-b96c-48e6-a3a2-d2bab50405f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:55.107794 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b86f7b-b96c-48e6-a3a2-d2bab50405f6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:55.108104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b86f7b-b96c-48e6-a3a2-d2bab50405f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:55.108370 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b86f7b-b96c-48e6-a3a2-d2bab50405f6 None admin] PATH_INFO: `/v3/projects/d43f38801344407291af57c99bc81066` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:55.112928 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b86f7b-b96c-48e6-a3a2-d2bab50405f6 None admin] RBAC: Authorizing `identity:get_project(project_id=d43f38801344407291af57c99bc81066)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:55.114422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b86f7b-b96c-48e6-a3a2-d2bab50405f6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:55.116553 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 184/378] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:54 2022] GET /identity/v3/projects/d43f38801344407291af57c99bc81066 => generated 344 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:55.122417 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21220248-2cdc-490b-8f9c-cf49e419e137 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:55.244929 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21220248-2cdc-490b-8f9c-cf49e419e137 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:55.281626 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b73672-c838-4ca3-8a1c-c7ec5ce1575f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:55.282300 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b73672-c838-4ca3-8a1c-c7ec5ce1575f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:55.282582 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b73672-c838-4ca3-8a1c-c7ec5ce1575f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:55.282859 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b73672-c838-4ca3-8a1c-c7ec5ce1575f None admin] PATH_INFO: `/v3/groups/a7224c5824fa4e0dac94113334f8854f` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:55.289526 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b73672-c838-4ca3-8a1c-c7ec5ce1575f None admin] RBAC: Authorizing `identity:get_group(group_id=a7224c5824fa4e0dac94113334f8854f)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:55.291020 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b73672-c838-4ca3-8a1c-c7ec5ce1575f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:55.295026 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 195/379] 10.222.0.48 () {56 vars in 1180 bytes} [Fri Sep 9 20:46:55 2022] GET /identity/v3/groups/a7224c5824fa4e0dac94113334f8854f => generated 229 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:55.298677 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b86f7b-b96c-48e6-a3a2-d2bab50405f6 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:55.410994 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:55.420485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b86f7b-b96c-48e6-a3a2-d2bab50405f6 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:55.472483 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be83fb1b-f62d-4ce9-90e4-7d95a579375c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:55.473107 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83fb1b-f62d-4ce9-90e4-7d95a579375c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:55.473385 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83fb1b-f62d-4ce9-90e4-7d95a579375c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:55.473652 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83fb1b-f62d-4ce9-90e4-7d95a579375c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:55.474394 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be83fb1b-f62d-4ce9-90e4-7d95a579375c None admin] RBAC: Adding query filter params (group.id=a7224c5824fa4e0dac94113334f8854f, role.id=fcbf519fbc84458f913671a5a1d0914c, scope.project.id=d43f38801344407291af57c99bc81066) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:55.474702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be83fb1b-f62d-4ce9-90e4-7d95a579375c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:55.476197 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be83fb1b-f62d-4ce9-90e4-7d95a579375c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:55.489320 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 185/380] 10.222.0.48 () {56 vars in 1376 bytes} [Fri Sep 9 20:46:55 2022] GET /identity/v3/role_assignments?group.id=a7224c5824fa4e0dac94113334f8854f&scope.project.id=d43f38801344407291af57c99bc81066&role.id=fcbf519fbc84458f913671a5a1d0914c => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:55.624373 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c3037-3ead-4808-ad29-44f02dca1334 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:55.624668 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c3037-3ead-4808-ad29-44f02dca1334 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:55.624834 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c3037-3ead-4808-ad29-44f02dca1334 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:55.626542 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 196/381] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:55.627731 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80e219-b74a-479d-9ede-4bb89519ef5b None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:55.628020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80e219-b74a-479d-9ede-4bb89519ef5b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:55.628325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80e219-b74a-479d-9ede-4bb89519ef5b None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:55.629642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 186/382] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:55.632194 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677e782-f18a-4c85-aba9-99e4e3cdfab0 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:55.632736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677e782-f18a-4c85-aba9-99e4e3cdfab0 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:55.633209 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677e782-f18a-4c85-aba9-99e4e3cdfab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:55.636159 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256e302-dfc6-4412-b811-72b5eb36510e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:55.636444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256e302-dfc6-4412-b811-72b5eb36510e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:55.636723 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256e302-dfc6-4412-b811-72b5eb36510e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:55.658957 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-6677e782-f18a-4c85-aba9-99e4e3cdfab0 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:55.660582 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-6677e782-f18a-4c85-aba9-99e4e3cdfab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:55.665575 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-3256e302-dfc6-4412-b811-72b5eb36510e None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:55.666810 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3256e302-dfc6-4412-b811-72b5eb36510e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:55.745112 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 197/383] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:55.745682 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:55.796530 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 187/384] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:46:56.059346 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6677e782-f18a-4c85-aba9-99e4e3cdfab0 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:56.174683 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256e302-dfc6-4412-b811-72b5eb36510e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:56.195072 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6677e782-f18a-4c85-aba9-99e4e3cdfab0 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:56.260817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35e17b-986b-49bc-87f6-ae0e8919be76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:56.261479 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35e17b-986b-49bc-87f6-ae0e8919be76 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:56.261758 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35e17b-986b-49bc-87f6-ae0e8919be76 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:56.262085 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35e17b-986b-49bc-87f6-ae0e8919be76 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:56.280963 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe35e17b-986b-49bc-87f6-ae0e8919be76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:56.285828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-fe35e17b-986b-49bc-87f6-ae0e8919be76 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:46:56.285828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:56.285828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:56.285828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:56.285828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:56.285828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:56.285828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:56.285828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:56.285828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:56.285828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:56.285828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:56.285828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:56.285828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:56.285828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:56.285828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:56.285828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:56.286752 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:56.286752 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:56.286752 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:56.286752 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:56.286752 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:56.286752 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:56.286752 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:56.286752 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:56.286752 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:56.286752 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:56.286752 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:56.286752 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:56.286752 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:56.286752 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:56.286752 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:56.286752 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:56.287886 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:56.287886 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:56.287886 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:56.287886 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:56.287886 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:56.287886 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:56.287886 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:56.287886 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:56.287886 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:56.287886 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:56.287886 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:46:56.287886 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:46:56.287886 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 198/385] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Sep 9 20:46:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:56.295911 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35e17b-986b-49bc-87f6-ae0e8919be76 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:56.299783 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:56.317042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256e302-dfc6-4412-b811-72b5eb36510e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:56.376410 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e973e-3920-4a91-af24-fb784a89d938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:56.377052 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e973e-3920-4a91-af24-fb784a89d938 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:56.377327 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e973e-3920-4a91-af24-fb784a89d938 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:56.377620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e973e-3920-4a91-af24-fb784a89d938 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:56.386688 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:56.401680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e973e-3920-4a91-af24-fb784a89d938 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:56.407489 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-f26e973e-3920-4a91-af24-fb784a89d938 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:56.407489 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:56.407489 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:56.407489 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:56.407489 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:56.407489 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:56.407489 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:56.407489 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:56.407489 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:56.407489 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:56.407489 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:56.407489 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:56.407489 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:56.407489 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:56.407489 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:56.407489 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:56.408456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:56.408456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:56.408456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:56.408456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:56.408456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:56.408456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:56.408456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:56.408456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:56.408456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:56.408456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:56.408456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:56.408456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:56.408456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:56.408456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:56.408456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:56.408456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:56.409665 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:56.409665 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:56.409665 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:56.409665 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:56.409665 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:56.409665 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:56.409665 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:56.409665 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:56.409665 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:56.409665 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:56.409665 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:56.409665 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:46:56.409665 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 188/386] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:56.415033 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e973e-3920-4a91-af24-fb784a89d938 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:56.498202 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35e17b-986b-49bc-87f6-ae0e8919be76 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:56.568172 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e973e-3920-4a91-af24-fb784a89d938 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:56.572772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07c4bc-d38d-43c5-bf92-fab933254cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:56.573398 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07c4bc-d38d-43c5-bf92-fab933254cbb None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:56.573677 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07c4bc-d38d-43c5-bf92-fab933254cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:56.573948 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07c4bc-d38d-43c5-bf92-fab933254cbb None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:56.574617 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e07c4bc-d38d-43c5-bf92-fab933254cbb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:56.574916 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e07c4bc-d38d-43c5-bf92-fab933254cbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:56.576311 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e07c4bc-d38d-43c5-bf92-fab933254cbb None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:56.587589 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 199/387] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:46:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:56.593144 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07c4bc-d38d-43c5-bf92-fab933254cbb None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:56.634152 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad2af9-31bc-40fa-902f-aa8986bf348a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:56.634764 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad2af9-31bc-40fa-902f-aa8986bf348a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:56.635039 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad2af9-31bc-40fa-902f-aa8986bf348a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:56.635306 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad2af9-31bc-40fa-902f-aa8986bf348a None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:56.635976 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad2af9-31bc-40fa-902f-aa8986bf348a None admin] RBAC: Adding query filter params (name=member) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:56.636306 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad2af9-31bc-40fa-902f-aa8986bf348a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:56.637691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad2af9-31bc-40fa-902f-aa8986bf348a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:56.651760 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 189/388] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Sep 9 20:46:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:46:56.658197 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad2af9-31bc-40fa-902f-aa8986bf348a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:56.681698 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:56.789471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07c4bc-d38d-43c5-bf92-fab933254cbb None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:56.804491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:56.861338 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cbbb1a-a891-46cf-ac79-876e3b690efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:56.863341 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbbb1a-a891-46cf-ac79-876e3b690efb None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:56.865203 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbbb1a-a891-46cf-ac79-876e3b690efb None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:56.866073 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbbb1a-a891-46cf-ac79-876e3b690efb None admin] PATH_INFO: `/v3/groups/2a4806378be54b8d9473e70d7eff9171` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:56.868832 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad2af9-31bc-40fa-902f-aa8986bf348a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:56.872104 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cbbb1a-a891-46cf-ac79-876e3b690efb None admin] RBAC: Authorizing `identity:get_group(group_id=2a4806378be54b8d9473e70d7eff9171)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:56.873930 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cbbb1a-a891-46cf-ac79-876e3b690efb None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:56.881718 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 200/389] 10.222.0.48 () {56 vars in 1180 bytes} [Fri Sep 9 20:46:56 2022] GET /identity/v3/groups/2a4806378be54b8d9473e70d7eff9171 => generated 226 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:56.884042 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cbbb1a-a891-46cf-ac79-876e3b690efb None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:56.946670 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d9a8f2-4c6c-4141-bf9d-f1a284f3fc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:56.947959 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9a8f2-4c6c-4141-bf9d-f1a284f3fc4e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:56.949561 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9a8f2-4c6c-4141-bf9d-f1a284f3fc4e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:56.949857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9a8f2-4c6c-4141-bf9d-f1a284f3fc4e None admin] PATH_INFO: `/v3/groups/2a4806378be54b8d9473e70d7eff9171` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:56.951971 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d9a8f2-4c6c-4141-bf9d-f1a284f3fc4e None admin] RBAC: Authorizing `identity:get_group(group_id=2a4806378be54b8d9473e70d7eff9171)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:56.953484 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d9a8f2-4c6c-4141-bf9d-f1a284f3fc4e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:56.955813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 190/390] 10.222.0.48 () {56 vars in 1180 bytes} [Fri Sep 9 20:46:56 2022] GET /identity/v3/groups/2a4806378be54b8d9473e70d7eff9171 => generated 226 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:56.956995 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f5e30-efce-4720-8c1c-568df8333a42 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:56.957123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f5e30-efce-4720-8c1c-568df8333a42 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:56.957288 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f5e30-efce-4720-8c1c-568df8333a42 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:56.958783 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 191/391] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:56.964160 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9f5e30-efce-4720-8c1c-568df8333a42 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:57.044141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cbbb1a-a891-46cf-ac79-876e3b690efb None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:57.073804 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:57.090630 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb64563-a492-494c-aa90-d6607c2816e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:57.091283 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb64563-a492-494c-aa90-d6607c2816e7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:57.091565 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb64563-a492-494c-aa90-d6607c2816e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:57.091840 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb64563-a492-494c-aa90-d6607c2816e7 None admin] PATH_INFO: `/v3/projects/1464b3dd76734fb593f3ed3582957e86` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:57.093240 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb64563-a492-494c-aa90-d6607c2816e7 None admin] RBAC: Authorizing `identity:get_project(project_id=1464b3dd76734fb593f3ed3582957e86)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:57.096057 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb64563-a492-494c-aa90-d6607c2816e7 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:57.098121 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 201/392] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:56 2022] GET /identity/v3/projects/1464b3dd76734fb593f3ed3582957e86 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:46:57.102992 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542eb108-c604-4c17-a6d0-490489b0bee2 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:57.102992 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542eb108-c604-4c17-a6d0-490489b0bee2 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:57.102992 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542eb108-c604-4c17-a6d0-490489b0bee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:57.112173 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9f5e30-efce-4720-8c1c-568df8333a42 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:57.121166 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-542eb108-c604-4c17-a6d0-490489b0bee2 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:57.124425 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-542eb108-c604-4c17-a6d0-490489b0bee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:57.154760 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7f755-00f2-406b-907c-fd5cbb03c1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:57.155740 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7f755-00f2-406b-907c-fd5cbb03c1f1 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:57.156801 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7f755-00f2-406b-907c-fd5cbb03c1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:57.157284 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7f755-00f2-406b-907c-fd5cbb03c1f1 None admin] PATH_INFO: `/v3/projects/1464b3dd76734fb593f3ed3582957e86` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:57.158641 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7f755-00f2-406b-907c-fd5cbb03c1f1 None admin] RBAC: Authorizing `identity:get_project(project_id=1464b3dd76734fb593f3ed3582957e86)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:57.160344 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7f755-00f2-406b-907c-fd5cbb03c1f1 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:57.162535 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 192/393] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:56 2022] GET /identity/v3/projects/1464b3dd76734fb593f3ed3582957e86 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:57.163612 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7f755-00f2-406b-907c-fd5cbb03c1f1 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:57.172343 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:57.215345 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 202/394] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:57.215950 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542eb108-c604-4c17-a6d0-490489b0bee2 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:57.332689 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542eb108-c604-4c17-a6d0-490489b0bee2 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:57.339220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7f755-00f2-406b-907c-fd5cbb03c1f1 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:57.370654 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a6771-5cd4-4bcc-abb0-e3a11b15c5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:57.371248 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a6771-5cd4-4bcc-abb0-e3a11b15c5e0 None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:57.371533 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a6771-5cd4-4bcc-abb0-e3a11b15c5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:57.371822 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a6771-5cd4-4bcc-abb0-e3a11b15c5e0 None admin] PATH_INFO: `/v3/projects/1464b3dd76734fb593f3ed3582957e86/groups/2a4806378be54b8d9473e70d7eff9171/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:57.373794 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a6771-5cd4-4bcc-abb0-e3a11b15c5e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=1464b3dd76734fb593f3ed3582957e86, group_id=2a4806378be54b8d9473e70d7eff9171, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:57.375239 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a6771-5cd4-4bcc-abb0-e3a11b15c5e0 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:57.401018 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a446b30-2745-4660-b6b2-65da97207cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:57.401799 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a446b30-2745-4660-b6b2-65da97207cf0 None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:57.402082 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a446b30-2745-4660-b6b2-65da97207cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:57.402361 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a446b30-2745-4660-b6b2-65da97207cf0 None admin] PATH_INFO: `/v3/projects/1464b3dd76734fb593f3ed3582957e86/groups/2a4806378be54b8d9473e70d7eff9171/roles/13072ad4b81649459f59b8137db492ec` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:57.404728 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a446b30-2745-4660-b6b2-65da97207cf0 None admin] RBAC: Authorizing `identity:create_grant(project_id=1464b3dd76734fb593f3ed3582957e86, group_id=2a4806378be54b8d9473e70d7eff9171, role_id=13072ad4b81649459f59b8137db492ec)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:57.406357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a446b30-2745-4660-b6b2-65da97207cf0 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:57.441123 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 203/395] 10.222.0.48 () {58 vars in 1442 bytes} [Fri Sep 9 20:46:57 2022] PUT /identity/v3/projects/1464b3dd76734fb593f3ed3582957e86/groups/2a4806378be54b8d9473e70d7eff9171/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:46:57.470413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 193/396] 10.222.0.48 () {58 vars in 1442 bytes} [Fri Sep 9 20:46:57 2022] PUT /identity/v3/projects/1464b3dd76734fb593f3ed3582957e86/groups/2a4806378be54b8d9473e70d7eff9171/roles/13072ad4b81649459f59b8137db492ec => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:46:57.598711 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a6771-5cd4-4bcc-abb0-e3a11b15c5e0 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:57.637367 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:57.725656 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a6771-5cd4-4bcc-abb0-e3a11b15c5e0 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:57.765467 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c62dc-bea8-4bd8-9ebe-7cfc0fadc228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:57.766517 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c62dc-bea8-4bd8-9ebe-7cfc0fadc228 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:57.767001 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c62dc-bea8-4bd8-9ebe-7cfc0fadc228 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:57.767641 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c62dc-bea8-4bd8-9ebe-7cfc0fadc228 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:57.779714 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c62dc-bea8-4bd8-9ebe-7cfc0fadc228 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:57.786037 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-6b7c62dc-bea8-4bd8-9ebe-7cfc0fadc228 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:57.786037 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:57.786037 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:57.786037 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:57.786037 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:57.786037 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:57.786037 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:57.786037 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:57.786037 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:57.786037 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:57.786037 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:57.786037 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:57.786037 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:57.786037 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:57.786037 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:57.786037 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:57.786957 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:57.786957 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:57.786957 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:57.786957 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:57.786957 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:57.786957 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:57.786957 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:57.786957 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:57.786957 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:57.786957 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:57.786957 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:57.786957 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:57.786957 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:57.786957 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:57.786957 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:57.786957 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:57.787764 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:57.787764 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:57.787764 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:57.787764 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:57.787764 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:57.787764 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:57.787764 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:57.787764 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:57.787764 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:57.787764 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:57.787764 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:46:57.787764 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:46:57.790579 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 204/397] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:57.796579 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a446b30-2745-4660-b6b2-65da97207cf0 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:57.800809 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:57.878604 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb5537f-fdfa-438d-8071-66a20e2e34e4 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:57.878948 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb5537f-fdfa-438d-8071-66a20e2e34e4 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:57.879153 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb5537f-fdfa-438d-8071-66a20e2e34e4 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:57.880681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 205/398] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:57.887603 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc97c71-dd92-4740-97f8-1f64e98a39d2 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:57.887835 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc97c71-dd92-4740-97f8-1f64e98a39d2 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:57.888107 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc97c71-dd92-4740-97f8-1f64e98a39d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:57.908271 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-ffc97c71-dd92-4740-97f8-1f64e98a39d2 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:57.909666 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ffc97c71-dd92-4740-97f8-1f64e98a39d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:57.923337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a446b30-2745-4660-b6b2-65da97207cf0 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:57.986424 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb6458-b9fa-498d-9448-3b77ab1fab1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:57.987405 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb6458-b9fa-498d-9448-3b77ab1fab1f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:57.987988 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb6458-b9fa-498d-9448-3b77ab1fab1f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:57.988541 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb6458-b9fa-498d-9448-3b77ab1fab1f None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:57.989475 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb6458-b9fa-498d-9448-3b77ab1fab1f None admin] RBAC: Adding query filter params (name=member) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:57.989970 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb6458-b9fa-498d-9448-3b77ab1fab1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:57.991544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb6458-b9fa-498d-9448-3b77ab1fab1f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:57.997258 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 206/399] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:58.012891 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 194/400] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Sep 9 20:46:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:58.018559 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc97c71-dd92-4740-97f8-1f64e98a39d2 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:58.051208 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:58.140221 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc97c71-dd92-4740-97f8-1f64e98a39d2 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:58.179116 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe182a-000d-4aba-93ef-2585a90e184e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:58.180341 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe182a-000d-4aba-93ef-2585a90e184e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:58.180624 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe182a-000d-4aba-93ef-2585a90e184e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:58.180900 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe182a-000d-4aba-93ef-2585a90e184e None admin] PATH_INFO: `/v3/groups/2a4806378be54b8d9473e70d7eff9171` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:58.182009 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe182a-000d-4aba-93ef-2585a90e184e None admin] RBAC: Authorizing `identity:get_group(group_id=2a4806378be54b8d9473e70d7eff9171)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:58.183439 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe182a-000d-4aba-93ef-2585a90e184e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:58.185753 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 207/401] 10.222.0.48 () {56 vars in 1180 bytes} [Fri Sep 9 20:46:58 2022] GET /identity/v3/groups/2a4806378be54b8d9473e70d7eff9171 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:58.191320 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb6458-b9fa-498d-9448-3b77ab1fab1f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:58.299385 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:58.341554 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb6458-b9fa-498d-9448-3b77ab1fab1f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:58.384516 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe182a-000d-4aba-93ef-2585a90e184e None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:58.388534 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206d962f-2097-4c5a-ab2f-c546dcacd64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:58.389180 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d962f-2097-4c5a-ab2f-c546dcacd64d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:58.392253 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d962f-2097-4c5a-ab2f-c546dcacd64d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:58.392253 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d962f-2097-4c5a-ab2f-c546dcacd64d None admin] PATH_INFO: `/v3/projects/e13bca418ede487ab7d1230e28403173` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:58.393793 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206d962f-2097-4c5a-ab2f-c546dcacd64d None admin] RBAC: Authorizing `identity:get_project(project_id=e13bca418ede487ab7d1230e28403173)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:58.395389 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206d962f-2097-4c5a-ab2f-c546dcacd64d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:58.400778 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 195/402] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:58 2022] GET /identity/v3/projects/e13bca418ede487ab7d1230e28403173 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:58.403260 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206d962f-2097-4c5a-ab2f-c546dcacd64d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:58.511265 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:58.522339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe182a-000d-4aba-93ef-2585a90e184e None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:58.551230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206d962f-2097-4c5a-ab2f-c546dcacd64d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:58.564147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df3797-c0e7-4c3d-a245-03069070ef34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:58.565605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df3797-c0e7-4c3d-a245-03069070ef34 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:58.566037 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df3797-c0e7-4c3d-a245-03069070ef34 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:58.566453 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df3797-c0e7-4c3d-a245-03069070ef34 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:58.579875 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df3797-c0e7-4c3d-a245-03069070ef34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:58.587291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-a4df3797-c0e7-4c3d-a245-03069070ef34 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:46:58.587291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:58.587291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:58.587291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:58.587291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:58.587291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:58.587291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:58.587291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:58.587291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:58.587291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:58.587291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:58.587291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:58.587291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:58.587291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:58.587291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:58.587291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:58.595251 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:58.595251 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:58.595251 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:58.595251 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:58.595251 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:58.595251 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:58.595251 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:58.595251 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:58.595251 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:58.595251 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:58.595251 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:58.595251 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:58.595251 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:58.595251 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:58.595251 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:58.595251 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:58.596591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:58.596591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:58.596591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:58.596591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:58.596591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:58.596591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:58.596591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:58.596591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:58.596591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:58.596591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:58.596591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:46:58.596591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:46:58.596591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 208/403] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Sep 9 20:46:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:58.596591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbec22-88f3-40cd-ab7e-ed0ee174ada3 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:58.597646 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbec22-88f3-40cd-ab7e-ed0ee174ada3 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:58.597646 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbec22-88f3-40cd-ab7e-ed0ee174ada3 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:58.600914 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 209/404] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:46:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:46:58.602471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bbec22-88f3-40cd-ab7e-ed0ee174ada3 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:58.604057 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887bffd-719f-4a78-bd32-4ef9425861ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:58.604765 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887bffd-719f-4a78-bd32-4ef9425861ad None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:58.605042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887bffd-719f-4a78-bd32-4ef9425861ad None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:58.605310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887bffd-719f-4a78-bd32-4ef9425861ad None admin] PATH_INFO: `/v3/projects/e13bca418ede487ab7d1230e28403173/groups/2a4806378be54b8d9473e70d7eff9171/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:58.607551 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887bffd-719f-4a78-bd32-4ef9425861ad None admin] RBAC: Authorizing `identity:create_grant(project_id=e13bca418ede487ab7d1230e28403173, group_id=2a4806378be54b8d9473e70d7eff9171, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:58.609332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887bffd-719f-4a78-bd32-4ef9425861ad None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:58.613157 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:58.683307 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 196/405] 10.222.0.48 () {58 vars in 1442 bytes} [Fri Sep 9 20:46:58 2022] PUT /identity/v3/projects/e13bca418ede487ab7d1230e28403173/groups/2a4806378be54b8d9473e70d7eff9171/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:46:58.685335 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf205f-b9d2-4d45-96b8-6c5a95cddbcd None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:58.685630 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf205f-b9d2-4d45-96b8-6c5a95cddbcd None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:58.685906 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf205f-b9d2-4d45-96b8-6c5a95cddbcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:58.710386 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-08bf205f-b9d2-4d45-96b8-6c5a95cddbcd None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:46:58.716110 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-08bf205f-b9d2-4d45-96b8-6c5a95cddbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:46:58.756319 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bbec22-88f3-40cd-ab7e-ed0ee174ada3 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:58.797374 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 197/406] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:46:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:46:58.826575 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22854984-8324-40ad-ac89-cc1694f37ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:58.827579 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22854984-8324-40ad-ac89-cc1694f37ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:58.828096 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22854984-8324-40ad-ac89-cc1694f37ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:58.828595 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22854984-8324-40ad-ac89-cc1694f37ac7 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:58.829462 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22854984-8324-40ad-ac89-cc1694f37ac7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:58.830015 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22854984-8324-40ad-ac89-cc1694f37ac7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:58.831672 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22854984-8324-40ad-ac89-cc1694f37ac7 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:58.846328 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 210/407] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:46:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:58.848049 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf205f-b9d2-4d45-96b8-6c5a95cddbcd None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:58.997073 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf205f-b9d2-4d45-96b8-6c5a95cddbcd None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:59.030147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:59.044776 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39816d69-7ab9-497b-a1f5-16e8e7ecb884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:59.045427 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39816d69-7ab9-497b-a1f5-16e8e7ecb884 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:59.045706 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39816d69-7ab9-497b-a1f5-16e8e7ecb884 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:59.045979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39816d69-7ab9-497b-a1f5-16e8e7ecb884 None admin] PATH_INFO: `/v3/groups/2a4806378be54b8d9473e70d7eff9171` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:59.047130 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39816d69-7ab9-497b-a1f5-16e8e7ecb884 None admin] RBAC: Authorizing `identity:get_group(group_id=2a4806378be54b8d9473e70d7eff9171)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:59.048613 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39816d69-7ab9-497b-a1f5-16e8e7ecb884 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:59.050574 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 198/408] 10.222.0.48 () {56 vars in 1180 bytes} [Fri Sep 9 20:46:58 2022] GET /identity/v3/groups/2a4806378be54b8d9473e70d7eff9171 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:59.056109 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22854984-8324-40ad-ac89-cc1694f37ac7 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:59.084409 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:59.133713 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39816d69-7ab9-497b-a1f5-16e8e7ecb884 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:59.239318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22854984-8324-40ad-ac89-cc1694f37ac7 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:59.294284 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39816d69-7ab9-497b-a1f5-16e8e7ecb884 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:59.294592 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d2c82-04f2-4358-9dc9-05bb2e6aacdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:59.295158 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d2c82-04f2-4358-9dc9-05bb2e6aacdc None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:59.295443 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d2c82-04f2-4358-9dc9-05bb2e6aacdc None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:59.295727 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d2c82-04f2-4358-9dc9-05bb2e6aacdc None admin] PATH_INFO: `/v3/projects/e13bca418ede487ab7d1230e28403173` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:59.296873 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13d2c82-04f2-4358-9dc9-05bb2e6aacdc None admin] RBAC: Authorizing `identity:get_project(project_id=e13bca418ede487ab7d1230e28403173)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:59.298305 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13d2c82-04f2-4358-9dc9-05bb2e6aacdc None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:59.304180 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 211/409] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:59 2022] GET /identity/v3/projects/e13bca418ede487ab7d1230e28403173 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:59.306444 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d2c82-04f2-4358-9dc9-05bb2e6aacdc None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:59.353653 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fcf904-350b-4b41-81f8-4e329b4c0b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:59.354349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcf904-350b-4b41-81f8-4e329b4c0b86 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:59.354586 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcf904-350b-4b41-81f8-4e329b4c0b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:59.354870 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcf904-350b-4b41-81f8-4e329b4c0b86 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:59.380986 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fcf904-350b-4b41-81f8-4e329b4c0b86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:46:59.394669 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-b0fcf904-350b-4b41-81f8-4e329b4c0b86 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:46:59.394669 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:46:59.394669 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:46:59.394669 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:46:59.394669 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:46:59.394669 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:46:59.394669 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:46:59.394669 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:46:59.394669 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:46:59.394669 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:46:59.394669 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:46:59.394669 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:46:59.394669 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:46:59.394669 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:46:59.394669 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:46:59.394669 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:46:59.395662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:46:59.395662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:46:59.395662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:46:59.395662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:46:59.395662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:46:59.395662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:46:59.395662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:46:59.395662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:46:59.395662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:46:59.395662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:46:59.395662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:46:59.395662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:46:59.395662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:46:59.395662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:46:59.395662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:46:59.395662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:46:59.400968 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:46:59.400968 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:46:59.400968 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:46:59.400968 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:46:59.400968 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:46:59.400968 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:46:59.400968 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:46:59.400968 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:46:59.400968 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:46:59.400968 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:46:59.400968 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:46:59.400968 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:46:59.400968 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 199/410] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:46:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:46:59.415884 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:59.417460 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fcf904-350b-4b41-81f8-4e329b4c0b86 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:59.438935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d2c82-04f2-4358-9dc9-05bb2e6aacdc None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:59.484529 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9b7f4-116c-42cb-a3ba-79c5133205a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:59.485634 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9b7f4-116c-42cb-a3ba-79c5133205a0 None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:59.486253 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9b7f4-116c-42cb-a3ba-79c5133205a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:59.486735 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9b7f4-116c-42cb-a3ba-79c5133205a0 None admin] PATH_INFO: `/v3/projects/e13bca418ede487ab7d1230e28403173/groups/2a4806378be54b8d9473e70d7eff9171/roles/13072ad4b81649459f59b8137db492ec` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:59.493860 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d9b7f4-116c-42cb-a3ba-79c5133205a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=e13bca418ede487ab7d1230e28403173, group_id=2a4806378be54b8d9473e70d7eff9171, role_id=13072ad4b81649459f59b8137db492ec)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:59.495601 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d9b7f4-116c-42cb-a3ba-79c5133205a0 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:59.546571 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:46:59.571205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 212/411] 10.222.0.48 () {58 vars in 1442 bytes} [Fri Sep 9 20:46:59 2022] PUT /identity/v3/projects/e13bca418ede487ab7d1230e28403173/groups/2a4806378be54b8d9473e70d7eff9171/roles/13072ad4b81649459f59b8137db492ec => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:46:59.581007 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fcf904-350b-4b41-81f8-4e329b4c0b86 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:59.639094 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e59234e-38b0-4276-8e0f-d609e209344d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:59.639728 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59234e-38b0-4276-8e0f-d609e209344d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:59.640008 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59234e-38b0-4276-8e0f-d609e209344d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:59.640305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59234e-38b0-4276-8e0f-d609e209344d None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:59.640986 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e59234e-38b0-4276-8e0f-d609e209344d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:46:59.641293 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e59234e-38b0-4276-8e0f-d609e209344d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:59.642745 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e59234e-38b0-4276-8e0f-d609e209344d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:59.656463 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 200/412] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:46:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:59.662136 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9b7f4-116c-42cb-a3ba-79c5133205a0 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:59.813910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9b7f4-116c-42cb-a3ba-79c5133205a0 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:46:59.848579 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344ad7a-18ef-46d7-9d3f-25af6c00e982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:46:59.849213 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344ad7a-18ef-46d7-9d3f-25af6c00e982 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:46:59.849491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344ad7a-18ef-46d7-9d3f-25af6c00e982 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:46:59.849777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344ad7a-18ef-46d7-9d3f-25af6c00e982 None admin] PATH_INFO: `/v3/groups/a7224c5824fa4e0dac94113334f8854f` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:46:59.850967 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5344ad7a-18ef-46d7-9d3f-25af6c00e982 None admin] RBAC: Authorizing `identity:get_group(group_id=a7224c5824fa4e0dac94113334f8854f)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:46:59.853027 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5344ad7a-18ef-46d7-9d3f-25af6c00e982 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:46:59.854881 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 213/413] 10.222.0.48 () {56 vars in 1180 bytes} [Fri Sep 9 20:46:59 2022] GET /identity/v3/groups/a7224c5824fa4e0dac94113334f8854f => generated 229 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:46:59.860372 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e59234e-38b0-4276-8e0f-d609e209344d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:46:59.997930 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e59234e-38b0-4276-8e0f-d609e209344d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:00.023723 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:00.049739 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf3c07-0163-43a1-94a5-823388b78b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:00.050398 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf3c07-0163-43a1-94a5-823388b78b9d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:00.050682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf3c07-0163-43a1-94a5-823388b78b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:00.050958 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf3c07-0163-43a1-94a5-823388b78b9d None admin] PATH_INFO: `/v3/projects/d43f38801344407291af57c99bc81066` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:00.053672 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaf3c07-0163-43a1-94a5-823388b78b9d None admin] RBAC: Authorizing `identity:get_project(project_id=d43f38801344407291af57c99bc81066)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:00.055301 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaf3c07-0163-43a1-94a5-823388b78b9d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:00.057401 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 201/414] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:46:59 2022] GET /identity/v3/projects/d43f38801344407291af57c99bc81066 => generated 344 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:00.063086 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344ad7a-18ef-46d7-9d3f-25af6c00e982 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:00.082159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:00.124964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15bccd-60a2-4d82-ba53-34e02bf5fcb2 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:00.125404 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15bccd-60a2-4d82-ba53-34e02bf5fcb2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:00.125794 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15bccd-60a2-4d82-ba53-34e02bf5fcb2 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:00.127242 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 202/415] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:00.132623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430225b-3c51-415d-b8d5-851be56690a5 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:00.132911 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430225b-3c51-415d-b8d5-851be56690a5 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:00.133193 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430225b-3c51-415d-b8d5-851be56690a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:00.152248 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b430225b-3c51-415d-b8d5-851be56690a5 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:00.153478 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b430225b-3c51-415d-b8d5-851be56690a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:00.207567 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344ad7a-18ef-46d7-9d3f-25af6c00e982 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:00.235114 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 203/416] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:47:00.241368 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbea93d-41a7-4e35-bddd-62fe052163f7 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:00.241937 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbea93d-41a7-4e35-bddd-62fe052163f7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:00.242301 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbea93d-41a7-4e35-bddd-62fe052163f7 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:00.247848 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 204/417] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:00.253672 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56036bf-df8d-44a0-b917-bdfe5e4226dc None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:00.254052 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56036bf-df8d-44a0-b917-bdfe5e4226dc None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:00.254361 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56036bf-df8d-44a0-b917-bdfe5e4226dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:00.255942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d4b89-488e-4009-97f4-2449c285f692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:00.256841 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d4b89-488e-4009-97f4-2449c285f692 None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:00.257331 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d4b89-488e-4009-97f4-2449c285f692 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:00.258323 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d4b89-488e-4009-97f4-2449c285f692 None admin] PATH_INFO: `/v3/projects/d43f38801344407291af57c99bc81066/groups/a7224c5824fa4e0dac94113334f8854f/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:00.261460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d4b89-488e-4009-97f4-2449c285f692 None admin] RBAC: Authorizing `identity:create_grant(project_id=d43f38801344407291af57c99bc81066, group_id=a7224c5824fa4e0dac94113334f8854f, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:00.264401 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d4b89-488e-4009-97f4-2449c285f692 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:00.283361 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f56036bf-df8d-44a0-b917-bdfe5e4226dc None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:00.284938 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f56036bf-df8d-44a0-b917-bdfe5e4226dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:00.364139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 205/418] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:47:00.371678 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 214/419] 10.222.0.48 () {58 vars in 1442 bytes} [Fri Sep 9 20:47:00 2022] PUT /identity/v3/projects/d43f38801344407291af57c99bc81066/groups/a7224c5824fa4e0dac94113334f8854f/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:47:00.517201 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56036bf-df8d-44a0-b917-bdfe5e4226dc None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:00.576182 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:00.650153 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56036bf-df8d-44a0-b917-bdfe5e4226dc None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:00.668390 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d4b89-488e-4009-97f4-2449c285f692 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:00.698543 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeaa270-6ebb-49a1-a12c-1275c124cd4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:00.699206 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeaa270-6ebb-49a1-a12c-1275c124cd4f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:00.699486 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeaa270-6ebb-49a1-a12c-1275c124cd4f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:00.700036 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeaa270-6ebb-49a1-a12c-1275c124cd4f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:00.715582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeaa270-6ebb-49a1-a12c-1275c124cd4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:47:00.720562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-8eeaa270-6ebb-49a1-a12c-1275c124cd4f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:47:00.720562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:00.720562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:00.720562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:00.720562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:00.720562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:00.720562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:00.720562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:00.720562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:00.720562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:00.720562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:00.720562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:00.720562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:47:00.720562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:47:00.720562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:47:00.720562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:47:00.721462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:47:00.721462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:47:00.721462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:47:00.721462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:47:00.721462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:00.721462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:00.721462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:00.721462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:00.721462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:00.721462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:00.721462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:00.721462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:00.721462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:00.721462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:00.721462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:00.721462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:00.722296 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:00.722296 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:00.722296 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:00.722296 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:00.722296 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:47:00.722296 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:47:00.722296 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:47:00.722296 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:47:00.722296 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:47:00.722296 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:47:00.722296 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:47:00.722296 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:00.722296 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 206/420] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Sep 9 20:47:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:00.729234 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeaa270-6ebb-49a1-a12c-1275c124cd4f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:00.773897 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:00.804330 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d4b89-488e-4009-97f4-2449c285f692 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:00.871651 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67efa4ed-d636-4f6e-8b28-07b40267ce74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:00.872319 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efa4ed-d636-4f6e-8b28-07b40267ce74 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:00.872596 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efa4ed-d636-4f6e-8b28-07b40267ce74 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:00.872861 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efa4ed-d636-4f6e-8b28-07b40267ce74 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:00.893774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67efa4ed-d636-4f6e-8b28-07b40267ce74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:47:00.898644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-67efa4ed-d636-4f6e-8b28-07b40267ce74 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:47:00.898644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:00.898644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:00.898644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:00.898644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:00.898644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:00.898644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:00.898644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:00.898644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:00.898644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:00.898644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:00.898644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:00.898644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:47:00.898644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:47:00.898644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:47:00.898644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:47:00.899496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:47:00.899496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:47:00.899496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:47:00.899496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:47:00.899496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:00.899496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:00.899496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:00.899496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:00.899496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:00.899496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:00.899496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:00.899496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:00.899496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:00.899496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:00.899496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:00.899496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:00.900450 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:00.900450 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:00.900450 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:00.900450 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:00.900450 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:47:00.900450 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:47:00.900450 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:47:00.900450 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:47:00.900450 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:47:00.900450 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:47:00.900450 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:47:00.900450 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:00.900450 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 215/421] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:47:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:00.909790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67efa4ed-d636-4f6e-8b28-07b40267ce74 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:00.958480 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeaa270-6ebb-49a1-a12c-1275c124cd4f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:00.960938 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:01.030793 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77942d-f3c1-46c5-bd6e-15fa2708a949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:01.031464 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77942d-f3c1-46c5-bd6e-15fa2708a949 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:01.031740 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77942d-f3c1-46c5-bd6e-15fa2708a949 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:01.032008 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77942d-f3c1-46c5-bd6e-15fa2708a949 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:01.032745 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e77942d-f3c1-46c5-bd6e-15fa2708a949 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:01.033045 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e77942d-f3c1-46c5-bd6e-15fa2708a949 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:01.034418 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e77942d-f3c1-46c5-bd6e-15fa2708a949 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:01.058258 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 207/422] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:47:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:01.064442 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77942d-f3c1-46c5-bd6e-15fa2708a949 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:01.109562 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67efa4ed-d636-4f6e-8b28-07b40267ce74 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:01.165227 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:01.177227 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed26afe-736e-446f-af4a-475b0ac4cdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:01.184806 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed26afe-736e-446f-af4a-475b0ac4cdf8 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:01.184806 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed26afe-736e-446f-af4a-475b0ac4cdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:01.184806 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed26afe-736e-446f-af4a-475b0ac4cdf8 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:01.184806 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed26afe-736e-446f-af4a-475b0ac4cdf8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:01.184806 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed26afe-736e-446f-af4a-475b0ac4cdf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:01.184806 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed26afe-736e-446f-af4a-475b0ac4cdf8 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:01.205116 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 216/423] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Sep 9 20:47:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:01.208584 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed26afe-736e-446f-af4a-475b0ac4cdf8 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:01.251816 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77942d-f3c1-46c5-bd6e-15fa2708a949 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:01.319701 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b728f-6816-4fa9-b24c-b2af7e5bd705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:01.320374 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b728f-6816-4fa9-b24c-b2af7e5bd705 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:01.320651 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b728f-6816-4fa9-b24c-b2af7e5bd705 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:01.320922 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b728f-6816-4fa9-b24c-b2af7e5bd705 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:01.387835 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:01.392732 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed26afe-736e-446f-af4a-475b0ac4cdf8 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:01.437800 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b9a3d-3c2e-4313-8a0b-8a333e52ee6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:01.438256 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b9a3d-3c2e-4313-8a0b-8a333e52ee6a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:01.438533 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b9a3d-3c2e-4313-8a0b-8a333e52ee6a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:01.438799 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b9a3d-3c2e-4313-8a0b-8a333e52ee6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:01.457095 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818b728f-6816-4fa9-b24c-b2af7e5bd705 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:01.458993 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818b728f-6816-4fa9-b24c-b2af7e5bd705 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:01.516099 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:01.527630 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93b9a3d-3c2e-4313-8a0b-8a333e52ee6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:01.529426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93b9a3d-3c2e-4313-8a0b-8a333e52ee6a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:01.599891 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 208/424] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:47:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:47:01.601564 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038675c-d40a-4603-8c40-c351017bd2ae None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:01.601854 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038675c-d40a-4603-8c40-c351017bd2ae None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:01.602131 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038675c-d40a-4603-8c40-c351017bd2ae None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:01.604327 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 209/425] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:01.609220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4038675c-d40a-4603-8c40-c351017bd2ae None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:01.652872 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 217/426] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:47:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 20:47:01.654461 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d8087-7900-4b42-8d45-a2be41deee6c None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:01.654585 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d8087-7900-4b42-8d45-a2be41deee6c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:01.654804 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d8087-7900-4b42-8d45-a2be41deee6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:01.675559 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-9d2d8087-7900-4b42-8d45-a2be41deee6c None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:01.676825 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-9d2d8087-7900-4b42-8d45-a2be41deee6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:01.702851 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:01.756095 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 218/427] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:47:01.756095 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d8087-7900-4b42-8d45-a2be41deee6c None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:01.766588 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4038675c-d40a-4603-8c40-c351017bd2ae None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:01.789511 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:01.832845 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0d6436-bb39-4917-8853-9b42b3391431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:01.833487 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d6436-bb39-4917-8853-9b42b3391431 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:01.833760 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d6436-bb39-4917-8853-9b42b3391431 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:01.834027 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d6436-bb39-4917-8853-9b42b3391431 None admin] PATH_INFO: `/v3/projects/1464b3dd76734fb593f3ed3582957e86` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:01.835485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0d6436-bb39-4917-8853-9b42b3391431 None admin] RBAC: Authorizing `identity:get_project(project_id=1464b3dd76734fb593f3ed3582957e86)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:01.837113 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0d6436-bb39-4917-8853-9b42b3391431 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:01.839026 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 210/428] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:47:01 2022] GET /identity/v3/projects/1464b3dd76734fb593f3ed3582957e86 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:01.845786 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0d6436-bb39-4917-8853-9b42b3391431 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:01.897489 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d8087-7900-4b42-8d45-a2be41deee6c None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:01.938132 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27438bb-b4e7-4c40-93cb-f46a5095e7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:01.939146 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27438bb-b4e7-4c40-93cb-f46a5095e7fc None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:01.939635 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27438bb-b4e7-4c40-93cb-f46a5095e7fc None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:01.940459 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27438bb-b4e7-4c40-93cb-f46a5095e7fc None admin] PATH_INFO: `/v3/projects/1464b3dd76734fb593f3ed3582957e86` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:01.941806 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27438bb-b4e7-4c40-93cb-f46a5095e7fc None admin] RBAC: Authorizing `identity:get_project(project_id=1464b3dd76734fb593f3ed3582957e86)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:01.943499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27438bb-b4e7-4c40-93cb-f46a5095e7fc None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:01.950370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 219/429] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:47:01 2022] GET /identity/v3/projects/1464b3dd76734fb593f3ed3582957e86 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:01.951842 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27438bb-b4e7-4c40-93cb-f46a5095e7fc None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:02.049361 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:02.063653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27438bb-b4e7-4c40-93cb-f46a5095e7fc None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:02.098812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0d6436-bb39-4917-8853-9b42b3391431 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:02.101021 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91520c9-6e08-4e37-8965-2ba5bc5c91bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:02.102426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91520c9-6e08-4e37-8965-2ba5bc5c91bc None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:02.102900 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91520c9-6e08-4e37-8965-2ba5bc5c91bc None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:02.103379 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91520c9-6e08-4e37-8965-2ba5bc5c91bc None admin] PATH_INFO: `/v3/groups/2a4806378be54b8d9473e70d7eff9171` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:02.104784 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91520c9-6e08-4e37-8965-2ba5bc5c91bc None admin] RBAC: Authorizing `identity:get_group(group_id=2a4806378be54b8d9473e70d7eff9171)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:02.106407 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91520c9-6e08-4e37-8965-2ba5bc5c91bc None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:02.112300 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 220/430] 10.222.0.48 () {56 vars in 1180 bytes} [Fri Sep 9 20:47:01 2022] GET /identity/v3/groups/2a4806378be54b8d9473e70d7eff9171 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:02.114646 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91520c9-6e08-4e37-8965-2ba5bc5c91bc None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:02.177128 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:02.183018 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff306d06-77a0-4417-8bae-cdabe08dc011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:02.183610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff306d06-77a0-4417-8bae-cdabe08dc011 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:02.183954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff306d06-77a0-4417-8bae-cdabe08dc011 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:02.184414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff306d06-77a0-4417-8bae-cdabe08dc011 None admin] PATH_INFO: `/v3/groups/2a4806378be54b8d9473e70d7eff9171` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:02.189273 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff306d06-77a0-4417-8bae-cdabe08dc011 None admin] RBAC: Authorizing `identity:get_group(group_id=2a4806378be54b8d9473e70d7eff9171)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:02.189273 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff306d06-77a0-4417-8bae-cdabe08dc011 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:02.189273 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 211/431] 10.222.0.48 () {56 vars in 1180 bytes} [Fri Sep 9 20:47:01 2022] GET /identity/v3/groups/2a4806378be54b8d9473e70d7eff9171 => generated 226 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:02.189750 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff306d06-77a0-4417-8bae-cdabe08dc011 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:02.232520 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91520c9-6e08-4e37-8965-2ba5bc5c91bc None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:02.272491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0efd76-4060-4594-a049-9f9f49ba5aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:02.273973 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0efd76-4060-4594-a049-9f9f49ba5aac None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:02.274319 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0efd76-4060-4594-a049-9f9f49ba5aac None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:02.274660 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0efd76-4060-4594-a049-9f9f49ba5aac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:02.275466 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0efd76-4060-4594-a049-9f9f49ba5aac None admin] RBAC: Adding query filter params (group.id=2a4806378be54b8d9473e70d7eff9171, role.id=c59d891d042e48bab625f7b3666ef536, scope.project.id=1464b3dd76734fb593f3ed3582957e86) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:02.275849 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0efd76-4060-4594-a049-9f9f49ba5aac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:02.277379 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0efd76-4060-4594-a049-9f9f49ba5aac None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:02.283392 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 221/432] 10.222.0.48 () {56 vars in 1376 bytes} [Fri Sep 9 20:47:02 2022] GET /identity/v3/role_assignments?group.id=2a4806378be54b8d9473e70d7eff9171&scope.project.id=1464b3dd76734fb593f3ed3582957e86&role.id=c59d891d042e48bab625f7b3666ef536 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:47:02.285321 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0efd76-4060-4594-a049-9f9f49ba5aac None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:02.293487 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff306d06-77a0-4417-8bae-cdabe08dc011 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:02.321470 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:02.337306 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afed20c7-5cee-4f02-a6dc-66679fa2e1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:02.338127 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed20c7-5cee-4f02-a6dc-66679fa2e1d2 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:02.338505 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed20c7-5cee-4f02-a6dc-66679fa2e1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:02.338900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed20c7-5cee-4f02-a6dc-66679fa2e1d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:02.355778 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afed20c7-5cee-4f02-a6dc-66679fa2e1d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:47:02.362569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-afed20c7-5cee-4f02-a6dc-66679fa2e1d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:47:02.362569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:02.362569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:02.362569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:02.362569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:02.362569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:02.362569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:02.362569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:02.362569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:02.362569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:02.362569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:02.362569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:02.362569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:47:02.362569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:47:02.362569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:47:02.362569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:47:02.363682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:47:02.363682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:47:02.363682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:47:02.363682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:47:02.363682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:02.363682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:02.363682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:02.363682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:02.363682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:02.363682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:02.363682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:02.363682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:02.363682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:02.363682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:02.363682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:02.363682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:02.364690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:02.364690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:02.364690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:02.364690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:02.364690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:47:02.364690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:47:02.364690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:47:02.364690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:47:02.364690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:47:02.364690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:47:02.364690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 20:47:02.364690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:02.372025 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 212/433] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:47:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:02.372025 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42501dd4-88e9-4502-bfa7-9a46c446b419 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:02.372025 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42501dd4-88e9-4502-bfa7-9a46c446b419 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:02.372025 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42501dd4-88e9-4502-bfa7-9a46c446b419 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:02.375245 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 213/434] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:02.376316 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42501dd4-88e9-4502-bfa7-9a46c446b419 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:02.430943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0efd76-4060-4594-a049-9f9f49ba5aac None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:02.458864 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:02.475268 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee90c6-577b-48c4-b244-33fbb6e279bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:02.476194 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee90c6-577b-48c4-b244-33fbb6e279bc None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:02.477156 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee90c6-577b-48c4-b244-33fbb6e279bc None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:02.477548 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee90c6-577b-48c4-b244-33fbb6e279bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:02.479184 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee90c6-577b-48c4-b244-33fbb6e279bc None admin] RBAC: Adding query filter params (group.id=2a4806378be54b8d9473e70d7eff9171, role.id=13072ad4b81649459f59b8137db492ec, scope.project.id=1464b3dd76734fb593f3ed3582957e86) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:02.479657 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee90c6-577b-48c4-b244-33fbb6e279bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:02.481279 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee90c6-577b-48c4-b244-33fbb6e279bc None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:02.491385 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 222/435] 10.222.0.48 () {56 vars in 1376 bytes} [Fri Sep 9 20:47:02 2022] GET /identity/v3/role_assignments?group.id=2a4806378be54b8d9473e70d7eff9171&scope.project.id=1464b3dd76734fb593f3ed3582957e86&role.id=13072ad4b81649459f59b8137db492ec => generated 611 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:02.492935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035f1a7-f9bb-48cb-a629-7776e8a204ac None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:02.493362 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035f1a7-f9bb-48cb-a629-7776e8a204ac None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:02.493847 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035f1a7-f9bb-48cb-a629-7776e8a204ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:02.508728 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42501dd4-88e9-4502-bfa7-9a46c446b419 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:02.515846 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-9035f1a7-f9bb-48cb-a629-7776e8a204ac None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:02.517539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-9035f1a7-f9bb-48cb-a629-7776e8a204ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:02.557119 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4102b738-84e5-43cc-8486-1b8a8e211c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:02.557981 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102b738-84e5-43cc-8486-1b8a8e211c5a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:02.558400 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102b738-84e5-43cc-8486-1b8a8e211c5a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:02.558821 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102b738-84e5-43cc-8486-1b8a8e211c5a None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:02.559636 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4102b738-84e5-43cc-8486-1b8a8e211c5a None admin] RBAC: Adding query filter params (name=member) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:02.560383 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4102b738-84e5-43cc-8486-1b8a8e211c5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:02.561943 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4102b738-84e5-43cc-8486-1b8a8e211c5a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:02.576302 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 214/436] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Sep 9 20:47:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:02.578593 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4102b738-84e5-43cc-8486-1b8a8e211c5a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:02.647528 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 223/437] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:47:02.727368 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4102b738-84e5-43cc-8486-1b8a8e211c5a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:02.757377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:02.769226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e7c24-f6e9-4e3a-af92-db96c5636594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:02.769840 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e7c24-f6e9-4e3a-af92-db96c5636594 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:02.779364 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e7c24-f6e9-4e3a-af92-db96c5636594 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:02.779364 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e7c24-f6e9-4e3a-af92-db96c5636594 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:02.856379 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e7c24-f6e9-4e3a-af92-db96c5636594 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:02.857770 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e7c24-f6e9-4e3a-af92-db96c5636594 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:02.862762 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f397e7-bf95-4c0b-b848-8e8de674b59d None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:02.863251 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f397e7-bf95-4c0b-b848-8e8de674b59d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:02.863325 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f397e7-bf95-4c0b-b848-8e8de674b59d None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:02.865209 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 224/438] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:02.871334 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80156ea4-5c35-4bbe-bb17-e94c2bfb3e1e None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:02.871895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80156ea4-5c35-4bbe-bb17-e94c2bfb3e1e None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:02.872403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80156ea4-5c35-4bbe-bb17-e94c2bfb3e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:02.897750 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-80156ea4-5c35-4bbe-bb17-e94c2bfb3e1e None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:02.899336 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-80156ea4-5c35-4bbe-bb17-e94c2bfb3e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:02.910844 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:02.989428 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 215/439] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:47:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:47:02.989428 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e7c24-f6e9-4e3a-af92-db96c5636594 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:02.990603 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 225/440] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:47:02.997038 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80156ea4-5c35-4bbe-bb17-e94c2bfb3e1e None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:03.028297 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:03.129085 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e7c24-f6e9-4e3a-af92-db96c5636594 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:03.155041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80156ea4-5c35-4bbe-bb17-e94c2bfb3e1e None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:03.172174 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7843e69-7411-4915-9637-e3e38ccbc80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:03.173007 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7843e69-7411-4915-9637-e3e38ccbc80c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:03.173163 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7843e69-7411-4915-9637-e3e38ccbc80c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:03.173534 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7843e69-7411-4915-9637-e3e38ccbc80c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:03.187139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7843e69-7411-4915-9637-e3e38ccbc80c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:47:03.193486 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-d7843e69-7411-4915-9637-e3e38ccbc80c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:47:03.193486 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:03.193486 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:03.193486 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:03.193486 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:03.193486 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:03.193486 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:03.193486 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:03.193486 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:03.193486 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:03.193486 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:03.193486 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:03.193486 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:47:03.193486 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:47:03.193486 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:47:03.193486 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:47:03.194480 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:47:03.194480 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:47:03.194480 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:47:03.194480 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:47:03.194480 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:03.194480 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:03.194480 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:03.194480 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:03.194480 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:03.194480 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:03.194480 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:03.194480 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:03.194480 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:03.194480 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:03.194480 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:03.194480 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:03.195309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:03.195309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:03.195309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:03.195309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:03.195309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:47:03.195309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:47:03.195309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:47:03.195309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:47:03.195309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:47:03.195309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:47:03.195309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:47:03.195309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:03.195309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 216/441] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Sep 9 20:47:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:03.201506 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84ba0c-7d23-42b9-abc6-63294fcce53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:03.201806 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7843e69-7411-4915-9637-e3e38ccbc80c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:03.202624 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84ba0c-7d23-42b9-abc6-63294fcce53a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:03.202966 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84ba0c-7d23-42b9-abc6-63294fcce53a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:03.203302 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84ba0c-7d23-42b9-abc6-63294fcce53a None admin] PATH_INFO: `/v3/projects/e13bca418ede487ab7d1230e28403173` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:03.204710 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d84ba0c-7d23-42b9-abc6-63294fcce53a None admin] RBAC: Authorizing `identity:get_project(project_id=e13bca418ede487ab7d1230e28403173)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:03.206214 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d84ba0c-7d23-42b9-abc6-63294fcce53a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:03.211317 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 226/442] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:47:02 2022] GET /identity/v3/projects/e13bca418ede487ab7d1230e28403173 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:03.218466 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84ba0c-7d23-42b9-abc6-63294fcce53a None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:03.240103 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:03.308729 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:03.327418 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7843e69-7411-4915-9637-e3e38ccbc80c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:03.340809 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84ba0c-7d23-42b9-abc6-63294fcce53a None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:03.363551 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69901c-2d56-4813-83d6-dfb36ddc44ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:03.364271 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69901c-2d56-4813-83d6-dfb36ddc44ad None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:03.364550 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69901c-2d56-4813-83d6-dfb36ddc44ad None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:03.364817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69901c-2d56-4813-83d6-dfb36ddc44ad None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:03.365481 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c69901c-2d56-4813-83d6-dfb36ddc44ad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:03.365783 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c69901c-2d56-4813-83d6-dfb36ddc44ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:03.367178 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c69901c-2d56-4813-83d6-dfb36ddc44ad None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:03.375152 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 217/443] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:47:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:47:03.376278 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69901c-2d56-4813-83d6-dfb36ddc44ad None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:03.378855 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b803d252-5063-4e8c-a9e3-6b47b9623dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:03.379576 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803d252-5063-4e8c-a9e3-6b47b9623dfe None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:03.379765 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803d252-5063-4e8c-a9e3-6b47b9623dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:03.380319 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803d252-5063-4e8c-a9e3-6b47b9623dfe None admin] PATH_INFO: `/v3/groups/2a4806378be54b8d9473e70d7eff9171` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:03.381324 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b803d252-5063-4e8c-a9e3-6b47b9623dfe None admin] RBAC: Authorizing `identity:get_group(group_id=2a4806378be54b8d9473e70d7eff9171)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:03.383264 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b803d252-5063-4e8c-a9e3-6b47b9623dfe None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:03.384891 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 227/444] 10.222.0.48 () {56 vars in 1180 bytes} [Fri Sep 9 20:47:03 2022] GET /identity/v3/groups/2a4806378be54b8d9473e70d7eff9171 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:03.385659 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b803d252-5063-4e8c-a9e3-6b47b9623dfe None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:03.472466 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:03.489906 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69901c-2d56-4813-83d6-dfb36ddc44ad None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:03.497478 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b803d252-5063-4e8c-a9e3-6b47b9623dfe None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:03.526376 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41b56f-f72f-4088-b445-7caaabbc1f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:03.527011 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41b56f-f72f-4088-b445-7caaabbc1f83 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:03.527291 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41b56f-f72f-4088-b445-7caaabbc1f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:03.527657 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41b56f-f72f-4088-b445-7caaabbc1f83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:03.539955 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf41b56f-f72f-4088-b445-7caaabbc1f83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:47:03.540587 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74962c24-5ee9-448e-8bcc-f86e333a37a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:03.541195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74962c24-5ee9-448e-8bcc-f86e333a37a0 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:03.541471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74962c24-5ee9-448e-8bcc-f86e333a37a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:03.541742 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74962c24-5ee9-448e-8bcc-f86e333a37a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:03.545972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-cf41b56f-f72f-4088-b445-7caaabbc1f83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:47:03.545972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:03.545972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:03.545972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:03.545972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:03.545972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:03.545972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:03.545972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:03.545972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:03.545972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:03.545972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:03.545972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:03.545972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:47:03.545972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:47:03.545972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:47:03.545972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:47:03.546880 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:47:03.546880 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:47:03.546880 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:47:03.546880 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:47:03.546880 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:03.546880 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:03.546880 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:03.546880 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:03.546880 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:03.546880 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:03.546880 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:03.546880 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:03.546880 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:03.546880 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:03.546880 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:03.546880 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:03.547671 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:03.547671 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:03.547671 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:03.547671 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:03.547671 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:47:03.547671 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:47:03.547671 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:47:03.547671 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:47:03.547671 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:47:03.547671 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:47:03.547671 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:47:03.547671 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:03.547671 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 218/445] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:47:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:03.548373 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41b56f-f72f-4088-b445-7caaabbc1f83 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:03.555661 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:03.613636 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74962c24-5ee9-448e-8bcc-f86e333a37a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:03.615148 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74962c24-5ee9-448e-8bcc-f86e333a37a0 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:03.658039 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41b56f-f72f-4088-b445-7caaabbc1f83 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:03.690774 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb0150-30d1-4691-827f-b19ba5042ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:03.691368 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb0150-30d1-4691-827f-b19ba5042ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:03.691644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb0150-30d1-4691-827f-b19ba5042ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:03.691919 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb0150-30d1-4691-827f-b19ba5042ce7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:03.692662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb0150-30d1-4691-827f-b19ba5042ce7 None admin] RBAC: Adding query filter params (group.id=2a4806378be54b8d9473e70d7eff9171, role.id=c59d891d042e48bab625f7b3666ef536, scope.project.id=e13bca418ede487ab7d1230e28403173) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:03.692970 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb0150-30d1-4691-827f-b19ba5042ce7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:03.694402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb0150-30d1-4691-827f-b19ba5042ce7 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:03.698336 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:03.701478 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 219/446] 10.222.0.48 () {56 vars in 1376 bytes} [Fri Sep 9 20:47:03 2022] GET /identity/v3/role_assignments?group.id=2a4806378be54b8d9473e70d7eff9171&scope.project.id=e13bca418ede487ab7d1230e28403173&role.id=c59d891d042e48bab625f7b3666ef536 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:03.702297 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb0150-30d1-4691-827f-b19ba5042ce7 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:03.723112 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 228/447] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:47:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:47:03.729317 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74962c24-5ee9-448e-8bcc-f86e333a37a0 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:03.781110 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:03.823067 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb0150-30d1-4691-827f-b19ba5042ce7 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:03.839311 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74962c24-5ee9-448e-8bcc-f86e333a37a0 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:03.860832 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed77c95-954e-4e2b-8124-a0456b40391c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:03.861578 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed77c95-954e-4e2b-8124-a0456b40391c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:03.861860 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed77c95-954e-4e2b-8124-a0456b40391c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:03.862137 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed77c95-954e-4e2b-8124-a0456b40391c None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:03.862818 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed77c95-954e-4e2b-8124-a0456b40391c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:03.863127 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed77c95-954e-4e2b-8124-a0456b40391c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:03.864788 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed77c95-954e-4e2b-8124-a0456b40391c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:03.874432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 220/448] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:47:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:03.879129 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d76a0e5-329e-4e3b-b839-a88c8e202cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:03.880110 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed77c95-954e-4e2b-8124-a0456b40391c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:03.880792 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76a0e5-329e-4e3b-b839-a88c8e202cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:03.881073 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76a0e5-329e-4e3b-b839-a88c8e202cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:03.881344 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76a0e5-329e-4e3b-b839-a88c8e202cd5 None admin] PATH_INFO: `/v3/projects/e13bca418ede487ab7d1230e28403173` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:03.883724 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d76a0e5-329e-4e3b-b839-a88c8e202cd5 None admin] RBAC: Authorizing `identity:get_project(project_id=e13bca418ede487ab7d1230e28403173)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:03.885203 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d76a0e5-329e-4e3b-b839-a88c8e202cd5 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:03.886826 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 229/449] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:47:03 2022] GET /identity/v3/projects/e13bca418ede487ab7d1230e28403173 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:03.892432 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d76a0e5-329e-4e3b-b839-a88c8e202cd5 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:03.951597 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:04.001551 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed77c95-954e-4e2b-8124-a0456b40391c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:04.017672 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d76a0e5-329e-4e3b-b839-a88c8e202cd5 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:04.032952 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:04.040056 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973d2cd-df08-4156-9b24-c3564b718b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:04.041527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973d2cd-df08-4156-9b24-c3564b718b51 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:04.041527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973d2cd-df08-4156-9b24-c3564b718b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:04.041527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973d2cd-df08-4156-9b24-c3564b718b51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:04.066046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d8c16-8039-459d-a54d-6d9e2a04be0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:04.066693 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d8c16-8039-459d-a54d-6d9e2a04be0c None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:04.066971 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d8c16-8039-459d-a54d-6d9e2a04be0c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:04.067242 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d8c16-8039-459d-a54d-6d9e2a04be0c None admin] PATH_INFO: `/v3/groups/2a4806378be54b8d9473e70d7eff9171` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:04.068411 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d8c16-8039-459d-a54d-6d9e2a04be0c None admin] RBAC: Authorizing `identity:get_group(group_id=2a4806378be54b8d9473e70d7eff9171)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:04.069899 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d8c16-8039-459d-a54d-6d9e2a04be0c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:04.071673 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 230/450] 10.222.0.48 () {56 vars in 1180 bytes} [Fri Sep 9 20:47:03 2022] GET /identity/v3/groups/2a4806378be54b8d9473e70d7eff9171 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:47:04.077531 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d8c16-8039-459d-a54d-6d9e2a04be0c None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:04.116542 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a973d2cd-df08-4156-9b24-c3564b718b51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:04.118791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a973d2cd-df08-4156-9b24-c3564b718b51 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:04.177142 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d8c16-8039-459d-a54d-6d9e2a04be0c None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:04.194043 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:04.217795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d233f-9a07-4d45-a700-259b02faabbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:04.217795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d233f-9a07-4d45-a700-259b02faabbd None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:04.217795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d233f-9a07-4d45-a700-259b02faabbd None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:04.217795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d233f-9a07-4d45-a700-259b02faabbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:04.218223 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d233f-9a07-4d45-a700-259b02faabbd None admin] RBAC: Adding query filter params (group.id=2a4806378be54b8d9473e70d7eff9171, role.id=13072ad4b81649459f59b8137db492ec, scope.project.id=e13bca418ede487ab7d1230e28403173) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:04.218223 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d233f-9a07-4d45-a700-259b02faabbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:04.218223 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d233f-9a07-4d45-a700-259b02faabbd None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:04.218492 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 221/451] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:47:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 20:47:04.220605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 231/452] 10.222.0.48 () {56 vars in 1376 bytes} [Fri Sep 9 20:47:04 2022] GET /identity/v3/role_assignments?group.id=2a4806378be54b8d9473e70d7eff9171&scope.project.id=e13bca418ede487ab7d1230e28403173&role.id=13072ad4b81649459f59b8137db492ec => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:47:04.224095 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973d2cd-df08-4156-9b24-c3564b718b51 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:04.256224 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:04.328906 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973d2cd-df08-4156-9b24-c3564b718b51 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:04.364091 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ae48b-16d3-4bcc-aa65-28a7bdef3c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:04.364739 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ae48b-16d3-4bcc-aa65-28a7bdef3c11 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:04.365013 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ae48b-16d3-4bcc-aa65-28a7bdef3c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:04.365281 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ae48b-16d3-4bcc-aa65-28a7bdef3c11 None admin] PATH_INFO: `/v3/projects/d43f38801344407291af57c99bc81066` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:04.366422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ae48b-16d3-4bcc-aa65-28a7bdef3c11 None admin] RBAC: Authorizing `identity:get_project(project_id=d43f38801344407291af57c99bc81066)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:04.368569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ae48b-16d3-4bcc-aa65-28a7bdef3c11 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:04.370206 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 222/453] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:47:04 2022] GET /identity/v3/projects/d43f38801344407291af57c99bc81066 => generated 344 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:47:04.375937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d233f-9a07-4d45-a700-259b02faabbd None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:04.479091 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d233f-9a07-4d45-a700-259b02faabbd None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:04.511525 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b552f931-f341-41f1-8744-cf3623039bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:04.512177 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552f931-f341-41f1-8744-cf3623039bdf None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:04.512456 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552f931-f341-41f1-8744-cf3623039bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:04.512731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552f931-f341-41f1-8744-cf3623039bdf None admin] PATH_INFO: `/v3/groups/a7224c5824fa4e0dac94113334f8854f` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:04.513817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b552f931-f341-41f1-8744-cf3623039bdf None admin] RBAC: Authorizing `identity:get_group(group_id=a7224c5824fa4e0dac94113334f8854f)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:04.515231 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b552f931-f341-41f1-8744-cf3623039bdf None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:04.516813 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 232/454] 10.222.0.48 () {56 vars in 1180 bytes} [Fri Sep 9 20:47:04 2022] GET /identity/v3/groups/a7224c5824fa4e0dac94113334f8854f => generated 229 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:04.522648 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ae48b-16d3-4bcc-aa65-28a7bdef3c11 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:04.619620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:04.623391 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ae48b-16d3-4bcc-aa65-28a7bdef3c11 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:04.656377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d537c-4a2e-47a7-9034-c23fae1d7fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:04.656975 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d537c-4a2e-47a7-9034-c23fae1d7fdc None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:04.657251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d537c-4a2e-47a7-9034-c23fae1d7fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:04.657520 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d537c-4a2e-47a7-9034-c23fae1d7fdc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:04.658235 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d537c-4a2e-47a7-9034-c23fae1d7fdc None admin] RBAC: Adding query filter params (group.id=a7224c5824fa4e0dac94113334f8854f, role.id=fcbf519fbc84458f913671a5a1d0914c, scope.project.id=d43f38801344407291af57c99bc81066) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:04.658540 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d537c-4a2e-47a7-9034-c23fae1d7fdc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:04.659983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d537c-4a2e-47a7-9034-c23fae1d7fdc None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:04.666065 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 223/455] 10.222.0.48 () {56 vars in 1376 bytes} [Fri Sep 9 20:47:04 2022] GET /identity/v3/role_assignments?group.id=a7224c5824fa4e0dac94113334f8854f&scope.project.id=d43f38801344407291af57c99bc81066&role.id=fcbf519fbc84458f913671a5a1d0914c => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:07.442968 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f7867-ae00-454a-b7ee-333b115a523b None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:07.443495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f7867-ae00-454a-b7ee-333b115a523b None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:07.443495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f7867-ae00-454a-b7ee-333b115a523b None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:07.444315 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e9e51-b54b-49b7-b3e2-7c12e7b84b8b None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:07.445696 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e9e51-b54b-49b7-b3e2-7c12e7b84b8b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:07.445800 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 233/456] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:07.446266 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e9e51-b54b-49b7-b3e2-7c12e7b84b8b None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:07.447749 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 224/457] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:07.450678 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22752a0-e793-4ef7-bd84-6d195f833bec None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:07.450958 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22752a0-e793-4ef7-bd84-6d195f833bec None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:07.451230 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22752a0-e793-4ef7-bd84-6d195f833bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:07.453613 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e80c1e-f954-4d8b-b2de-0e9c83729d80 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:07.454101 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e80c1e-f954-4d8b-b2de-0e9c83729d80 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:07.454585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e80c1e-f954-4d8b-b2de-0e9c83729d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:07.482906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-d22752a0-e793-4ef7-bd84-6d195f833bec None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:07.484107 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d22752a0-e793-4ef7-bd84-6d195f833bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:07.487051 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f7e80c1e-f954-4d8b-b2de-0e9c83729d80 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:07.488234 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f7e80c1e-f954-4d8b-b2de-0e9c83729d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:07.576539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:07.613696 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 234/458] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:47:07.613916 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 225/459] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:47:07.838724 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e25b3a-a589-4303-920c-7d32492bcf2e None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:07.839386 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e25b3a-a589-4303-920c-7d32492bcf2e None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:07.839996 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e25b3a-a589-4303-920c-7d32492bcf2e None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:07.841822 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 235/460] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:07.847112 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44759750-595b-4b9b-8c01-59c55beed020 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:07.847610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44759750-595b-4b9b-8c01-59c55beed020 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:07.848109 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44759750-595b-4b9b-8c01-59c55beed020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:07.867941 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-44759750-595b-4b9b-8c01-59c55beed020 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:07.869474 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-44759750-595b-4b9b-8c01-59c55beed020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:07.918929 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e25b3a-a589-4303-920c-7d32492bcf2e None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:07.976680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 226/461] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:47:07.977595 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44759750-595b-4b9b-8c01-59c55beed020 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:08.030346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:08.123345 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44759750-595b-4b9b-8c01-59c55beed020 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:08.126503 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e25b3a-a589-4303-920c-7d32492bcf2e None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:08.191278 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc610d5-0b5a-431e-b0c7-9ec39311f171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:08.193042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc610d5-0b5a-431e-b0c7-9ec39311f171 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:08.194054 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc610d5-0b5a-431e-b0c7-9ec39311f171 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:08.194883 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c1407-1234-455a-bdbc-7d925cae4e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:08.195381 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc610d5-0b5a-431e-b0c7-9ec39311f171 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:08.196218 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c1407-1234-455a-bdbc-7d925cae4e5b None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:08.196737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c1407-1234-455a-bdbc-7d925cae4e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:08.197213 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c1407-1234-455a-bdbc-7d925cae4e5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:08.208409 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0c1407-1234-455a-bdbc-7d925cae4e5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:08.210225 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0c1407-1234-455a-bdbc-7d925cae4e5b None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:08.224960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc610d5-0b5a-431e-b0c7-9ec39311f171 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:08.227033 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc610d5-0b5a-431e-b0c7-9ec39311f171 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:08.228677 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-7a0c1407-1234-455a-bdbc-7d925cae4e5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:08.228677 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:08.228677 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:08.228677 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:08.228677 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:08.228677 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:08.228677 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:08.228677 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:08.228677 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:08.228677 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:08.228677 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:08.228677 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:08.228677 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:08.228677 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:08.228677 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:08.228677 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:08.229568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:08.229568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:08.229568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:08.229568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:08.229568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:08.229568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:08.229568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:08.229568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:08.229568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:08.229568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:08.229568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:08.229568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:08.229568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:08.229568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:08.229568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:08.229568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:08.229568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:08.230422 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:08.230422 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:08.230422 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:08.232874 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 236/462] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:08.241390 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c1407-1234-455a-bdbc-7d925cae4e5b None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:08.248115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-6cc610d5-0b5a-431e-b0c7-9ec39311f171 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:08.248115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:08.248115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:08.248115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:08.248115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:08.248115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:08.248115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:08.248115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:08.248115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:08.248115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:08.248115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:08.248115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:08.248115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:08.248115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:08.248115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:08.248115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:08.249079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:08.249079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:08.249079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:08.249079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:08.249079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:08.249079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:08.249079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:08.249079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:08.249079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:08.249079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:08.249079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:08.249079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:08.249079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:08.249079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:08.249079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:08.249079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:08.249079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:08.249971 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:08.249971 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:08.249971 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:08.251905 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 227/463] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:08.275737 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc610d5-0b5a-431e-b0c7-9ec39311f171 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:08.291671 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:08.373737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c1407-1234-455a-bdbc-7d925cae4e5b None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:08.386868 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:08.411238 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbce869-b446-47ce-90e7-04a4e29564b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:08.411825 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbce869-b446-47ce-90e7-04a4e29564b9 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:08.411948 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc610d5-0b5a-431e-b0c7-9ec39311f171 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:08.412165 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbce869-b446-47ce-90e7-04a4e29564b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:08.412395 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbce869-b446-47ce-90e7-04a4e29564b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:08.413060 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbce869-b446-47ce-90e7-04a4e29564b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:08.413360 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbce869-b446-47ce-90e7-04a4e29564b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:08.415436 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbce869-b446-47ce-90e7-04a4e29564b9 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:08.433993 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 237/464] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:08.433993 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbce869-b446-47ce-90e7-04a4e29564b9 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:08.455687 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc9684-9318-4a67-8b26-ebd5baaefe81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:08.457503 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc9684-9318-4a67-8b26-ebd5baaefe81 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:08.457958 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc9684-9318-4a67-8b26-ebd5baaefe81 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:08.458455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc9684-9318-4a67-8b26-ebd5baaefe81 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:08.459131 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc9684-9318-4a67-8b26-ebd5baaefe81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:08.459447 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc9684-9318-4a67-8b26-ebd5baaefe81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:08.461458 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc9684-9318-4a67-8b26-ebd5baaefe81 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:08.474613 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 228/465] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:47:08.474613 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9c1cf-71c0-4096-abb8-58983975c8e6 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:08.476866 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9c1cf-71c0-4096-abb8-58983975c8e6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:08.476938 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9c1cf-71c0-4096-abb8-58983975c8e6 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:08.479210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 229/466] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:08.479210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9c1cf-71c0-4096-abb8-58983975c8e6 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:08.535028 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:08.544740 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbce869-b446-47ce-90e7-04a4e29564b9 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:08.579167 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930524e7-ad7e-4810-8303-00228e8a5ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:08.579779 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930524e7-ad7e-4810-8303-00228e8a5ecb None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:08.580055 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930524e7-ad7e-4810-8303-00228e8a5ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:08.580357 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930524e7-ad7e-4810-8303-00228e8a5ecb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:08.584888 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9c1cf-71c0-4096-abb8-58983975c8e6 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:08.590481 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930524e7-ad7e-4810-8303-00228e8a5ecb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:08.591928 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930524e7-ad7e-4810-8303-00228e8a5ecb None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:08.606637 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-930524e7-ad7e-4810-8303-00228e8a5ecb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:08.606637 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:08.606637 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:08.606637 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:08.606637 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:08.606637 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:08.606637 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:08.606637 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:08.606637 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:08.606637 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:08.606637 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:08.606637 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:08.606637 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:08.606637 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:08.606637 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:08.606637 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:08.607565 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:08.607565 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:08.607565 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:08.607565 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:08.607565 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:08.607565 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:08.607565 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:08.607565 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:08.607565 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:08.607565 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:08.607565 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:08.607565 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:08.607565 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:08.607565 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:08.607565 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:08.607565 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:08.607565 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:08.608459 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:08.608459 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:08.608459 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:08.608459 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 238/467] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:08.608671 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930524e7-ad7e-4810-8303-00228e8a5ecb None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:08.620323 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f3bb4-5b22-4915-aab3-29a6fd47a68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:08.620928 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f3bb4-5b22-4915-aab3-29a6fd47a68a None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:08.621207 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f3bb4-5b22-4915-aab3-29a6fd47a68a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:08.621473 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f3bb4-5b22-4915-aab3-29a6fd47a68a None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:08.622111 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f3bb4-5b22-4915-aab3-29a6fd47a68a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:08.623553 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f3bb4-5b22-4915-aab3-29a6fd47a68a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:08.631514 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:08.701883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 230/468] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Sep 9 20:47:08 2022] POST /identity/v3/users => generated 248 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:47:08.703099 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ea0d6-c4fe-424c-908a-c66b2bb976a3 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:08.703382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ea0d6-c4fe-424c-908a-c66b2bb976a3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:08.703653 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ea0d6-c4fe-424c-908a-c66b2bb976a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:08.715542 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930524e7-ad7e-4810-8303-00228e8a5ecb None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:08.724884 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-370ea0d6-c4fe-424c-908a-c66b2bb976a3 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:08.726142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-370ea0d6-c4fe-424c-908a-c66b2bb976a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:08.757185 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca896a90-9e78-40a5-8d3b-ac0e50b34b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:08.759046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca896a90-9e78-40a5-8d3b-ac0e50b34b42 None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:08.759995 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca896a90-9e78-40a5-8d3b-ac0e50b34b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:08.760745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca896a90-9e78-40a5-8d3b-ac0e50b34b42 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:08.761828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca896a90-9e78-40a5-8d3b-ac0e50b34b42 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:08.763676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca896a90-9e78-40a5-8d3b-ac0e50b34b42 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:08.800254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 231/469] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:47:08.809655 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f98e56-abb3-4f0d-af46-42b90e8af0c1 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:08.810289 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f98e56-abb3-4f0d-af46-42b90e8af0c1 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:08.810547 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f98e56-abb3-4f0d-af46-42b90e8af0c1 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:08.813128 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 232/470] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:08 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:08.813128 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f98e56-abb3-4f0d-af46-42b90e8af0c1 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:08.828834 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:08.840353 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 239/471] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Sep 9 20:47:08 2022] POST /identity/v3/users => generated 246 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:47:08.842379 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e986823-565f-42b5-b6bc-86f913ead79e None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:08.842894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e986823-565f-42b5-b6bc-86f913ead79e None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:08.843352 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e986823-565f-42b5-b6bc-86f913ead79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:08.885787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-2e986823-565f-42b5-b6bc-86f913ead79e None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:08.885787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2e986823-565f-42b5-b6bc-86f913ead79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:08.973849 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 240/472] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:47:08.980644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f98e56-abb3-4f0d-af46-42b90e8af0c1 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:09.002336 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:09.019085 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33065dc-1596-40f8-9cc5-9bd7ea69c37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:09.019711 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33065dc-1596-40f8-9cc5-9bd7ea69c37b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:09.019989 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33065dc-1596-40f8-9cc5-9bd7ea69c37b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:09.020352 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33065dc-1596-40f8-9cc5-9bd7ea69c37b None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:09.021070 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33065dc-1596-40f8-9cc5-9bd7ea69c37b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:09.021426 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33065dc-1596-40f8-9cc5-9bd7ea69c37b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:09.022838 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33065dc-1596-40f8-9cc5-9bd7ea69c37b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:09.035694 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 233/473] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:09.041613 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e986823-565f-42b5-b6bc-86f913ead79e None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:09.087956 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33065dc-1596-40f8-9cc5-9bd7ea69c37b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:09.153621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e986823-565f-42b5-b6bc-86f913ead79e None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:09.185236 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:09.194104 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bf0b9-c25f-40f2-a2e8-f7db6d548ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:09.204002 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bf0b9-c25f-40f2-a2e8-f7db6d548ecf None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:09.204002 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bf0b9-c25f-40f2-a2e8-f7db6d548ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:09.204002 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bf0b9-c25f-40f2-a2e8-f7db6d548ecf None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:09.204002 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5bf0b9-c25f-40f2-a2e8-f7db6d548ecf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:09.204002 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5bf0b9-c25f-40f2-a2e8-f7db6d548ecf None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:09.216921 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33065dc-1596-40f8-9cc5-9bd7ea69c37b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:09.257917 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3092741-16f4-409a-b819-1fcedec5615e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:09.258556 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3092741-16f4-409a-b819-1fcedec5615e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:09.258834 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3092741-16f4-409a-b819-1fcedec5615e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:09.259104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3092741-16f4-409a-b819-1fcedec5615e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:09.270860 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3092741-16f4-409a-b819-1fcedec5615e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:09.272445 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3092741-16f4-409a-b819-1fcedec5615e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:09.290787 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-f3092741-16f4-409a-b819-1fcedec5615e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:09.290787 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:09.290787 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:09.290787 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:09.290787 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:09.290787 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:09.290787 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:09.290787 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:09.290787 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:09.290787 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:09.290787 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:09.290787 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:09.290787 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:09.290787 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:09.290787 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:09.290787 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:09.291725 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:09.291725 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:09.291725 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:09.291725 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:09.291725 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:09.291725 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:09.291725 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:09.291725 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:09.291725 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:09.291725 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:09.291725 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:09.291725 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:09.291725 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:09.291725 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:09.291725 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:09.291725 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:09.291725 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:09.292648 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:09.292648 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:09.292648 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:09.292648 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 234/474] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:47:09.292860 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3092741-16f4-409a-b819-1fcedec5615e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:09.312014 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 241/475] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Sep 9 20:47:09 2022] POST /identity/v3/users => generated 248 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:47:09.317028 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bf0b9-c25f-40f2-a2e8-f7db6d548ecf None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:09.334461 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:09.448676 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3092741-16f4-409a-b819-1fcedec5615e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:09.478813 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bf0b9-c25f-40f2-a2e8-f7db6d548ecf None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:09.496205 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1ffc1-2e42-4e68-966b-67178566f0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:09.497541 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1ffc1-2e42-4e68-966b-67178566f0e0 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:09.498068 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1ffc1-2e42-4e68-966b-67178566f0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:09.499183 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1ffc1-2e42-4e68-966b-67178566f0e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:09.500099 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc1ffc1-2e42-4e68-966b-67178566f0e0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:09.501729 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc1ffc1-2e42-4e68-966b-67178566f0e0 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:09.536353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.sql.core [None req-4fc1ffc1-2e42-4e68-966b-67178566f0e0 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 20:47:09.536353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 09 20:47:09.536353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [parameters: {'id': 'fecbbb28578141f7b8f929e7cfb17db6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 09 20:47:09.536353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81629) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 20:47:09.560862 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d4711-97a8-4219-8d80-e310cceda76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:09.561535 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d4711-97a8-4219-8d80-e310cceda76e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:09.561823 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d4711-97a8-4219-8d80-e310cceda76e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:09.562095 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d4711-97a8-4219-8d80-e310cceda76e None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:09.562755 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d4711-97a8-4219-8d80-e310cceda76e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:09.563124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d4711-97a8-4219-8d80-e310cceda76e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:09.564609 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d4711-97a8-4219-8d80-e310cceda76e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:09.565510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-4fc1ffc1-2e42-4e68-966b-67178566f0e0 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 09 20:47:09.565510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:09.565510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 20:47:09.565510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 20:47:09.565510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 20:47:09.565510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 20:47:09.565510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 20:47:09.565510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 20:47:09.565510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 20:47:09.565510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application conn.query(q) Sep 09 20:47:09.565510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 20:47:09.565510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 20:47:09.565510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 20:47:09.565510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application result.read() Sep 09 20:47:09.565510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 20:47:09.566409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 20:47:09.566409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 20:47:09.566409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 20:47:09.566409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 20:47:09.566409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 20:47:09.566409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 20:47:09.566409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 20:47:09.566409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 20:47:09.566409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:09.566409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 20:47:09.566409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:09.566409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:09.566409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 20:47:09.566409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 20:47:09.566409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 09 20:47:09.566409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return ref.to_dict() Sep 09 20:47:09.566409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 20:47:09.566409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application next(self.gen) Sep 09 20:47:09.567346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 20:47:09.567346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application yield resource Sep 09 20:47:09.567346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 20:47:09.567346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application next(self.gen) Sep 09 20:47:09.567346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 20:47:09.567346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 20:47:09.567346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 20:47:09.567346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 20:47:09.567346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 20:47:09.567346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise self.value Sep 09 20:47:09.567346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 20:47:09.567346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 20:47:09.567346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 20:47:09.567346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application session.commit() Sep 09 20:47:09.567346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 20:47:09.567346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 20:47:09.567346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 20:47:09.568268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 20:47:09.568268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 20:47:09.568268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self.session.flush() Sep 09 20:47:09.568268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 20:47:09.568268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 20:47:09.568268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 20:47:09.568268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 20:47:09.568268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 20:47:09.568268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application compat.raise_( Sep 09 20:47:09.568268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 20:47:09.568268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception Sep 09 20:47:09.568268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 20:47:09.568268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 20:47:09.568268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 20:47:09.568268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 20:47:09.568268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 20:47:09.568268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 20:47:09.569184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 20:47:09.569184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 20:47:09.569184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 09 20:47:09.569184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 09 20:47:09.569184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 20:47:09.569184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 20:47:09.569184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 20:47:09.569184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 20:47:09.569184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 20:47:09.569184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 20:47:09.569184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 20:47:09.569184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 20:47:09.569184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 20:47:09.569184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 20:47:09.569184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 20:47:09.569184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception Sep 09 20:47:09.570046 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 20:47:09.570046 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 20:47:09.570046 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 20:47:09.570046 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 20:47:09.570046 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 20:47:09.570046 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 20:47:09.570046 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 20:47:09.570046 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application conn.query(q) Sep 09 20:47:09.570046 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 20:47:09.570046 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 20:47:09.570046 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 20:47:09.570046 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application result.read() Sep 09 20:47:09.570046 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 20:47:09.570046 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 20:47:09.570046 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 20:47:09.570046 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 20:47:09.570820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 20:47:09.570820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 20:47:09.570820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 20:47:09.570820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 20:47:09.570820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 20:47:09.570820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 09 20:47:09.570820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [parameters: {'id': 'fecbbb28578141f7b8f929e7cfb17db6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 09 20:47:09.570820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 20:47:09.570820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:09.570820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:47:09.570820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:09.570820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:09.570820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:09.570820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:09.570820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:09.570820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:09.571614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:09.571614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:09.571614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:09.571614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:09.571614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:09.571614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:09.571614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 09 20:47:09.571614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 09 20:47:09.571614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:09.571614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:09.571614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 09 20:47:09.571614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 09 20:47:09.571614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 20:47:09.571614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 20:47:09.571614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 09 20:47:09.571614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:09.576807 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 235/476] 10.222.0.48 () {60 vars in 1131 bytes} [Fri Sep 9 20:47:09 2022] POST /identity/v3/roles => generated 144 bytes in 284 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:09.579938 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 242/477] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:09.583442 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1ffc1-2e42-4e68-966b-67178566f0e0 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:09.585279 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d4711-97a8-4219-8d80-e310cceda76e None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:09.647104 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:09.710021 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:09.716892 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1ffc1-2e42-4e68-966b-67178566f0e0 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:09.727661 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d4711-97a8-4219-8d80-e310cceda76e None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:09.756318 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ed988-3e24-40b7-90a1-a1db1b3b46f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:09.757355 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ed988-3e24-40b7-90a1-a1db1b3b46f1 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:09.757789 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ed988-3e24-40b7-90a1-a1db1b3b46f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:09.758231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ed988-3e24-40b7-90a1-a1db1b3b46f1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:09.770568 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ed988-3e24-40b7-90a1-a1db1b3b46f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:47:09.771511 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7488b9-295f-4844-aebe-9cbc0c47f906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:09.772172 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7488b9-295f-4844-aebe-9cbc0c47f906 None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:09.772458 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7488b9-295f-4844-aebe-9cbc0c47f906 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:09.772732 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7488b9-295f-4844-aebe-9cbc0c47f906 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:09.773934 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7488b9-295f-4844-aebe-9cbc0c47f906 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:09.775469 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7488b9-295f-4844-aebe-9cbc0c47f906 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:09.777940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-938ed988-3e24-40b7-90a1-a1db1b3b46f1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:47:09.777940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:09.777940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:09.777940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:09.777940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:09.777940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:09.777940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:09.777940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:09.777940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:09.777940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:09.777940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:09.777940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:09.777940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:47:09.777940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:47:09.777940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:47:09.777940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:47:09.778805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:47:09.778805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:47:09.778805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:47:09.778805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:47:09.778805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:09.778805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:09.778805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:09.778805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:09.778805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:09.778805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:09.778805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:09.778805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:09.778805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:09.778805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:09.778805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:09.778805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:09.779643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:09.779643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:09.779643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:09.779643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:09.779643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:47:09.779643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:47:09.779643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:47:09.779643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:47:09.779643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:47:09.779643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:47:09.779643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 20:47:09.779643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:09.787856 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 236/478] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Sep 9 20:47:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:09.789971 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ed988-3e24-40b7-90a1-a1db1b3b46f1 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:09.884248 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 243/479] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Sep 9 20:47:09 2022] POST /identity/v3/users => generated 249 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:47:09.909036 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ed988-3e24-40b7-90a1-a1db1b3b46f1 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:09.959838 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bbbe09-6425-4131-9568-89d7614daa4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:09.960977 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bbbe09-6425-4131-9568-89d7614daa4b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:09.961456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bbbe09-6425-4131-9568-89d7614daa4b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:09.961930 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bbbe09-6425-4131-9568-89d7614daa4b None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:09.962835 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bbbe09-6425-4131-9568-89d7614daa4b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:09.963352 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bbbe09-6425-4131-9568-89d7614daa4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:09.965132 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bbbe09-6425-4131-9568-89d7614daa4b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:09.981267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 237/480] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:47:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:11.550583 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3444ac4-2161-4b50-9bcb-1cc644efde4c None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:11.551107 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3444ac4-2161-4b50-9bcb-1cc644efde4c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:11.551107 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3444ac4-2161-4b50-9bcb-1cc644efde4c None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:11.552879 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 244/481] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:11.558254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e667d3-dd06-477b-a986-e548f0d55ba4 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:11.558706 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e667d3-dd06-477b-a986-e548f0d55ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:11.558783 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e667d3-dd06-477b-a986-e548f0d55ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:11.590559 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-54e667d3-dd06-477b-a986-e548f0d55ba4 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:11.591765 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-54e667d3-dd06-477b-a986-e548f0d55ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:11.608549 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:11.679682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 238/482] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:47:11.832374 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bbb6a-ebdb-4600-a4f1-1aa58e4e704c None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:11.832683 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bbb6a-ebdb-4600-a4f1-1aa58e4e704c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:11.832836 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bbb6a-ebdb-4600-a4f1-1aa58e4e704c None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:11.834284 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 245/483] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:11.845320 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942e724-8692-4f2b-a261-9319f8de6c3b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:11.845609 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942e724-8692-4f2b-a261-9319f8de6c3b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:11.845886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942e724-8692-4f2b-a261-9319f8de6c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:11.869052 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f942e724-8692-4f2b-a261-9319f8de6c3b None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:11.870244 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f942e724-8692-4f2b-a261-9319f8de6c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:11.953014 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 239/484] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:47:11.970354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973bbb6a-ebdb-4600-a4f1-1aa58e4e704c None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:12.074736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:12.079154 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973bbb6a-ebdb-4600-a4f1-1aa58e4e704c None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:12.093093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555018d8-93eb-4258-a9d5-d107f343f0a6 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:12.093093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555018d8-93eb-4258-a9d5-d107f343f0a6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:12.093093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555018d8-93eb-4258-a9d5-d107f343f0a6 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:12.093093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 240/485] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:12.093093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5a976-7b28-4656-a12d-d2120c7089f0 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:12.093093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5a976-7b28-4656-a12d-d2120c7089f0 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:12.093618 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5a976-7b28-4656-a12d-d2120c7089f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:12.114116 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e689c96-3dc8-474a-b90d-465652536d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:12.114738 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e689c96-3dc8-474a-b90d-465652536d1e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:12.115010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e689c96-3dc8-474a-b90d-465652536d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:12.115276 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e689c96-3dc8-474a-b90d-465652536d1e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:12.128054 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e689c96-3dc8-474a-b90d-465652536d1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:47:12.133626 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f3d5a976-7b28-4656-a12d-d2120c7089f0 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:12.133895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-5e689c96-3dc8-474a-b90d-465652536d1e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:12.133895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:12.133895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:12.133895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:12.133895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:12.133895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:12.133895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:12.133895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:12.133895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:12.133895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:12.133895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:12.133895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:12.133895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:47:12.133895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:47:12.133895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:47:12.133895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:47:12.134735 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:47:12.134735 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:47:12.134735 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:47:12.134735 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:47:12.134735 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:12.134735 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:12.134735 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:12.134735 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:12.134735 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:12.134735 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:12.134735 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:12.134735 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:12.134735 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:12.134735 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:12.134735 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:12.134735 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:12.135544 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:12.135544 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:12.135544 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:12.135544 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:12.135544 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:47:12.135544 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:47:12.135544 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:47:12.135544 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:47:12.135544 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:47:12.135544 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:47:12.135544 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:12.135544 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:12.136185 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f3d5a976-7b28-4656-a12d-d2120c7089f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:12.139927 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 246/486] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Sep 9 20:47:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:12.140909 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e689c96-3dc8-474a-b90d-465652536d1e None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:12.254798 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e689c96-3dc8-474a-b90d-465652536d1e None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:12.297895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33c81d-3aad-4047-8e3a-672d7c6e7cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:12.298508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33c81d-3aad-4047-8e3a-672d7c6e7cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:12.298783 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33c81d-3aad-4047-8e3a-672d7c6e7cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:12.299055 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33c81d-3aad-4047-8e3a-672d7c6e7cc9 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:12.299727 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33c81d-3aad-4047-8e3a-672d7c6e7cc9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:12.300029 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33c81d-3aad-4047-8e3a-672d7c6e7cc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:12.301422 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33c81d-3aad-4047-8e3a-672d7c6e7cc9 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:12.318971 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 241/487] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:47:12.321530 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 247/488] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Sep 9 20:47:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:12.329346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5a976-7b28-4656-a12d-d2120c7089f0 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:12.330649 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33c81d-3aad-4047-8e3a-672d7c6e7cc9 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:12.340482 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:12.379177 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:12.485754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33c81d-3aad-4047-8e3a-672d7c6e7cc9 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:12.520523 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5a976-7b28-4656-a12d-d2120c7089f0 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:12.541034 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a86843-b7b9-47fd-b142-a8f03575fd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:12.542086 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a86843-b7b9-47fd-b142-a8f03575fd03 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:12.542580 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a86843-b7b9-47fd-b142-a8f03575fd03 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:12.543059 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a86843-b7b9-47fd-b142-a8f03575fd03 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:12.565585 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a86843-b7b9-47fd-b142-a8f03575fd03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:47:12.571663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-17a86843-b7b9-47fd-b142-a8f03575fd03 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:12.571663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:12.571663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:12.571663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:12.571663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:12.571663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:12.571663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:12.571663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:12.571663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:12.571663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:12.571663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:12.571663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:12.571663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:47:12.571663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:47:12.571663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:47:12.571663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:47:12.572582 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:47:12.572582 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:47:12.572582 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:47:12.572582 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:47:12.572582 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:12.572582 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:12.572582 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:12.572582 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:12.572582 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:12.572582 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:12.572582 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:12.572582 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:12.572582 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:12.572582 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:12.572582 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:12.572582 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:12.573394 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:12.573394 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:12.573394 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:12.573394 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:12.573394 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:47:12.573394 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:47:12.573394 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:47:12.573394 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:47:12.573394 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:47:12.573394 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:47:12.573394 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:12.573394 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:12.576524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 248/489] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Sep 9 20:47:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:12.583056 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a86843-b7b9-47fd-b142-a8f03575fd03 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:12.621180 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25616f3c-af76-4e0d-9018-3088bcd008b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:12.622661 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25616f3c-af76-4e0d-9018-3088bcd008b4 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:12.623082 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25616f3c-af76-4e0d-9018-3088bcd008b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:12.623545 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25616f3c-af76-4e0d-9018-3088bcd008b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:12.638927 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25616f3c-af76-4e0d-9018-3088bcd008b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:12.641173 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25616f3c-af76-4e0d-9018-3088bcd008b4 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:12.661002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-25616f3c-af76-4e0d-9018-3088bcd008b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:12.661002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:12.661002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:12.661002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:12.661002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:12.661002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:12.661002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:12.661002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:12.661002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:12.661002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:12.661002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:12.661002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:12.661002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:12.661002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:12.661002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:12.661002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:12.661960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:12.661960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:12.661960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:12.661960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:12.661960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:12.661960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:12.661960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:12.661960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:12.661960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:12.661960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:12.661960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:12.661960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:12.661960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:12.661960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:12.661960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:12.661960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:12.661960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:12.662814 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:12.662814 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:12.662814 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:12.662814 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 242/490] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:12.663033 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25616f3c-af76-4e0d-9018-3088bcd008b4 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:12.679574 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:12.697980 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a86843-b7b9-47fd-b142-a8f03575fd03 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:12.728683 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:12.735965 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31ff99-6149-4c77-adfb-ba883790680f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:12.736616 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31ff99-6149-4c77-adfb-ba883790680f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:12.736941 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31ff99-6149-4c77-adfb-ba883790680f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:12.737157 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31ff99-6149-4c77-adfb-ba883790680f None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:12.737818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31ff99-6149-4c77-adfb-ba883790680f None admin] RBAC: Adding query filter params (name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:12.738119 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31ff99-6149-4c77-adfb-ba883790680f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:12.739477 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31ff99-6149-4c77-adfb-ba883790680f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:12.747724 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 249/491] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Sep 9 20:47:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:47:12.753327 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72235d7-27d0-4aef-96cc-e37bce3208b6 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:12.753609 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72235d7-27d0-4aef-96cc-e37bce3208b6 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:12.753881 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72235d7-27d0-4aef-96cc-e37bce3208b6 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:12.755301 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 250/492] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:12.755785 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72235d7-27d0-4aef-96cc-e37bce3208b6 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:12.779572 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25616f3c-af76-4e0d-9018-3088bcd008b4 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:12.815543 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a450155-d6f2-4018-96d3-0e5b26996438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:12.816246 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a450155-d6f2-4018-96d3-0e5b26996438 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:12.816495 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a450155-d6f2-4018-96d3-0e5b26996438 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:12.816764 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a450155-d6f2-4018-96d3-0e5b26996438 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:12.828246 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a450155-d6f2-4018-96d3-0e5b26996438 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:47:12.833471 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-0a450155-d6f2-4018-96d3-0e5b26996438 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:12.833471 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:12.833471 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:12.833471 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:12.833471 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:12.833471 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:12.833471 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:12.833471 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:12.833471 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:12.833471 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:12.833471 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:12.833471 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:12.833471 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:47:12.833471 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:47:12.833471 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:47:12.833471 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:47:12.834359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:47:12.834359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:47:12.834359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:47:12.834359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:47:12.834359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:12.834359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:12.834359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:12.834359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:12.834359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:12.834359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:12.834359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:12.834359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:12.834359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:12.834359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:12.834359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:12.834359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:12.835151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:12.835151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:12.835151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:12.835151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:12.835151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:47:12.835151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:47:12.835151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:47:12.835151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:47:12.835151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:47:12.835151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:47:12.835151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:12.835151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:12.835151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 243/493] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Sep 9 20:47:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:12.836392 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a450155-d6f2-4018-96d3-0e5b26996438 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:12.869669 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72235d7-27d0-4aef-96cc-e37bce3208b6 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:12.907994 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be996d-8993-4b5d-8010-154b03a941d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:12.908974 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be996d-8993-4b5d-8010-154b03a941d6 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:12.909320 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be996d-8993-4b5d-8010-154b03a941d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:12.909651 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be996d-8993-4b5d-8010-154b03a941d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:12.910315 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be996d-8993-4b5d-8010-154b03a941d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:12.910615 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be996d-8993-4b5d-8010-154b03a941d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:12.912271 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be996d-8993-4b5d-8010-154b03a941d6 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:12.925671 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 251/494] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:12.930621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344f71b-31cc-4900-9130-e7e86cd9d448 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:12.930621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344f71b-31cc-4900-9130-e7e86cd9d448 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:12.930621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344f71b-31cc-4900-9130-e7e86cd9d448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:12.935895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:12.948192 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-b344f71b-31cc-4900-9130-e7e86cd9d448 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:12.950960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a450155-d6f2-4018-96d3-0e5b26996438 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:12.951079 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b344f71b-31cc-4900-9130-e7e86cd9d448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:12.972262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:12.989834 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4931d6-19ff-4a3a-98f6-16bd500b0ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:12.990466 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4931d6-19ff-4a3a-98f6-16bd500b0ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:12.992860 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4931d6-19ff-4a3a-98f6-16bd500b0ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:12.992860 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4931d6-19ff-4a3a-98f6-16bd500b0ab4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:13.002037 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4931d6-19ff-4a3a-98f6-16bd500b0ab4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:13.003589 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4931d6-19ff-4a3a-98f6-16bd500b0ab4 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:13.019978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-1e4931d6-19ff-4a3a-98f6-16bd500b0ab4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:13.019978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:13.019978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:13.019978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:13.019978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:13.019978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:13.019978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:13.019978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:13.019978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:13.019978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:13.019978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:13.019978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:13.019978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:13.019978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:13.019978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:13.019978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:13.021097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:13.021097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:13.021097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:13.021097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:13.021097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:13.021097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:13.021097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:13.021097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:13.021097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:13.021097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:13.021097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:13.021097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:13.021097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:13.021097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:13.021097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:13.021097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:13.021097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:13.022136 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:13.022136 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:13.022136 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:13.022136 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 244/495] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:13.022136 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4931d6-19ff-4a3a-98f6-16bd500b0ab4 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:13.027570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 252/496] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:47:13.028484 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344f71b-31cc-4900-9130-e7e86cd9d448 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:13.132976 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344f71b-31cc-4900-9130-e7e86cd9d448 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:13.134525 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4931d6-19ff-4a3a-98f6-16bd500b0ab4 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:13.168912 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc28f2-e487-49a1-a06b-0474d69035f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:13.169668 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9fa1e-5707-4d56-895e-ad82064c2fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:13.170100 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc28f2-e487-49a1-a06b-0474d69035f7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:13.170297 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9fa1e-5707-4d56-895e-ad82064c2fe1 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:13.170556 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9fa1e-5707-4d56-895e-ad82064c2fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:13.170833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9fa1e-5707-4d56-895e-ad82064c2fe1 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:13.171180 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc28f2-e487-49a1-a06b-0474d69035f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:13.171500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9fa1e-5707-4d56-895e-ad82064c2fe1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:13.171807 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9fa1e-5707-4d56-895e-ad82064c2fe1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:13.172041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc28f2-e487-49a1-a06b-0474d69035f7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:13.173228 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9fa1e-5707-4d56-895e-ad82064c2fe1 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:13.174464 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:13.181707 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fc28f2-e487-49a1-a06b-0474d69035f7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:13.187179 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 245/497] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Sep 9 20:47:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:13.187317 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fc28f2-e487-49a1-a06b-0474d69035f7 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:13.187405 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecafee-de5f-41ed-9ac2-75cc8b094304 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:13.187405 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecafee-de5f-41ed-9ac2-75cc8b094304 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:13.187405 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecafee-de5f-41ed-9ac2-75cc8b094304 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:13.189624 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 246/498] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:13.190293 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fecafee-de5f-41ed-9ac2-75cc8b094304 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:13.204190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-29fc28f2-e487-49a1-a06b-0474d69035f7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 20:47:13.204190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:13.204190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:13.204190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:13.204190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:13.204190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:13.204190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:13.204190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:13.204190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:13.204190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:13.204190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:13.204190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:13.204190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:47:13.204190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:47:13.204190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:47:13.204190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:47:13.205134 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:13.205134 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:13.205134 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:47:13.205134 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:13.205134 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:47:13.205134 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:13.205134 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:13.205134 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:13.205134 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:13.205134 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:13.205134 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:13.205134 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:13.205134 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:13.205134 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:13.205134 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:13.205134 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:13.205134 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:13.205134 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:13.206017 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:13.206017 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:13.206017 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:47:13.206017 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:47:13.206017 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:47:13.206017 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:47:13.206017 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:47:13.206017 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:47:13.206017 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 20:47:13.206017 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:13.207817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 253/499] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:47:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:13.208637 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc28f2-e487-49a1-a06b-0474d69035f7 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:13.227355 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:13.294891 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fecafee-de5f-41ed-9ac2-75cc8b094304 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:13.313399 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc28f2-e487-49a1-a06b-0474d69035f7 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:13.330042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6c959-7f26-4210-a16b-3660bf58fc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:13.330646 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6c959-7f26-4210-a16b-3660bf58fc93 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:13.330925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6c959-7f26-4210-a16b-3660bf58fc93 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:13.331199 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6c959-7f26-4210-a16b-3660bf58fc93 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:13.331837 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6c959-7f26-4210-a16b-3660bf58fc93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:13.332174 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6c959-7f26-4210-a16b-3660bf58fc93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:13.333563 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6c959-7f26-4210-a16b-3660bf58fc93 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:13.346599 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 247/500] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:13.346851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864a767-16b1-4215-b138-91819e89e667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:13.347483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864a767-16b1-4215-b138-91819e89e667 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:13.347761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f019d8f-61b6-40e5-8526-ce175e970a40 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:13.347856 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864a767-16b1-4215-b138-91819e89e667 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:13.348044 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f019d8f-61b6-40e5-8526-ce175e970a40 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:13.348178 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864a767-16b1-4215-b138-91819e89e667 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:13.348355 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f019d8f-61b6-40e5-8526-ce175e970a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:13.359912 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5864a767-16b1-4215-b138-91819e89e667 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:13.361402 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5864a767-16b1-4215-b138-91819e89e667 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:13.365634 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-2f019d8f-61b6-40e5-8526-ce175e970a40 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:13.367376 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2f019d8f-61b6-40e5-8526-ce175e970a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:13.375902 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-5864a767-16b1-4215-b138-91819e89e667 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:13.375902 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:13.375902 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:13.375902 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:13.375902 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:13.375902 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:13.375902 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:13.375902 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:13.375902 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:13.375902 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:13.375902 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:13.375902 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:13.375902 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:13.375902 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:13.375902 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:13.375902 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:13.376986 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:13.376986 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:13.376986 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:13.376986 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:13.376986 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:13.376986 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:13.376986 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:13.376986 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:13.376986 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:13.376986 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:13.376986 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:13.376986 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:13.376986 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:13.376986 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:13.376986 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:13.376986 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:13.376986 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:13.378058 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:13.378058 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:13.378058 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:13.378058 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 254/501] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:13.378058 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864a767-16b1-4215-b138-91819e89e667 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:13.422550 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:13.435861 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 248/502] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:47:13.436546 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f019d8f-61b6-40e5-8526-ce175e970a40 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:13.462413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:13.478010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864a767-16b1-4215-b138-91819e89e667 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:13.515216 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd1d39-fd67-4d47-b9f0-118fcf245451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:13.515828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd1d39-fd67-4d47-b9f0-118fcf245451 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:13.516144 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd1d39-fd67-4d47-b9f0-118fcf245451 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:13.516422 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd1d39-fd67-4d47-b9f0-118fcf245451 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:13.517292 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcd1d39-fd67-4d47-b9f0-118fcf245451 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:13.518057 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcd1d39-fd67-4d47-b9f0-118fcf245451 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:13.519851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcd1d39-fd67-4d47-b9f0-118fcf245451 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:13.542523 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f019d8f-61b6-40e5-8526-ce175e970a40 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:13.566986 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 255/503] 10.222.0.48 () {56 vars in 1137 bytes} [Fri Sep 9 20:47:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:13.567828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd1d39-fd67-4d47-b9f0-118fcf245451 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:13.579963 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b142337-ba0e-4ed9-90d1-181611d1348e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:13.580635 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b142337-ba0e-4ed9-90d1-181611d1348e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:13.580971 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b142337-ba0e-4ed9-90d1-181611d1348e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:13.581247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b142337-ba0e-4ed9-90d1-181611d1348e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:13.594826 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b142337-ba0e-4ed9-90d1-181611d1348e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:47:13.599812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-5b142337-ba0e-4ed9-90d1-181611d1348e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:13.599812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:13.599812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:13.599812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:13.599812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:13.599812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:13.599812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:13.599812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:13.599812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:13.599812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:13.599812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:13.599812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:13.599812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:47:13.599812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:47:13.599812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:47:13.599812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:47:13.600751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:47:13.600751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:47:13.600751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:47:13.600751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:47:13.600751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:13.600751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:13.600751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:13.600751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:13.600751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:13.600751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:13.600751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:13.600751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:13.600751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:13.600751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:13.600751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:13.600751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:13.602318 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:13.602318 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:13.602318 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:13.602318 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:13.602318 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:47:13.602318 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:47:13.602318 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:47:13.602318 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:47:13.602318 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:47:13.602318 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:47:13.602318 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:13.602318 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:13.602318 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 249/504] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Sep 9 20:47:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:13.603280 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b142337-ba0e-4ed9-90d1-181611d1348e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:13.678703 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd1d39-fd67-4d47-b9f0-118fcf245451 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:13.681494 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:13.695718 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:13.711970 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b142337-ba0e-4ed9-90d1-181611d1348e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:13.714670 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb32de-d96c-484b-b0d4-c714ecfa133b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:13.715300 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb32de-d96c-484b-b0d4-c714ecfa133b None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:13.715577 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb32de-d96c-484b-b0d4-c714ecfa133b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:13.715853 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb32de-d96c-484b-b0d4-c714ecfa133b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:13.723210 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb32de-d96c-484b-b0d4-c714ecfa133b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:13.724747 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb32de-d96c-484b-b0d4-c714ecfa133b None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:13.736274 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-d4cb32de-d96c-484b-b0d4-c714ecfa133b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:47:13.736274 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:13.736274 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:13.736274 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:13.736274 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:13.736274 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:13.736274 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:13.736274 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:13.736274 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:13.736274 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:13.736274 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:13.736274 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:13.736274 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:47:13.736274 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:47:13.736274 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:47:13.736274 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:47:13.737176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:13.737176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:13.737176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:47:13.737176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:13.737176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:47:13.737176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:13.737176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:13.737176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:13.737176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:13.737176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:13.737176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:13.737176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:13.737176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:13.737176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:13.737176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:13.737176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:13.737176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:13.737176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:13.743520 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:13.743520 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:13.743520 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:47:13.743520 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:47:13.743520 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:47:13.743520 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:47:13.743520 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:47:13.743520 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:47:13.743520 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:47:13.743520 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:13.743520 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 256/505] 10.222.0.48 () {56 vars in 1093 bytes} [Fri Sep 9 20:47:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:13.743520 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb32de-d96c-484b-b0d4-c714ecfa133b None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:13.747612 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f10273-932f-4a9f-9235-0b14ba43e0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:13.748481 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f10273-932f-4a9f-9235-0b14ba43e0a1 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:13.749200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f10273-932f-4a9f-9235-0b14ba43e0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:13.749475 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f10273-932f-4a9f-9235-0b14ba43e0a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:13.750131 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f10273-932f-4a9f-9235-0b14ba43e0a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:13.750438 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f10273-932f-4a9f-9235-0b14ba43e0a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:13.751813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f10273-932f-4a9f-9235-0b14ba43e0a1 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:13.763200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 250/506] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:13.763981 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f10273-932f-4a9f-9235-0b14ba43e0a1 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:13.837825 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb32de-d96c-484b-b0d4-c714ecfa133b None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:13.866386 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f10273-932f-4a9f-9235-0b14ba43e0a1 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:13.871146 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04429174-4f52-4206-b3d3-4496663d48a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:13.871841 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04429174-4f52-4206-b3d3-4496663d48a2 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:13.872143 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04429174-4f52-4206-b3d3-4496663d48a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:13.872411 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04429174-4f52-4206-b3d3-4496663d48a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:13.899314 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6ec8b-9207-4ec3-b153-5ae82037e822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:13.899917 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6ec8b-9207-4ec3-b153-5ae82037e822 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:13.900230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6ec8b-9207-4ec3-b153-5ae82037e822 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:13.900505 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6ec8b-9207-4ec3-b153-5ae82037e822 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:13.901170 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f6ec8b-9207-4ec3-b153-5ae82037e822 None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:13.901472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f6ec8b-9207-4ec3-b153-5ae82037e822 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:13.902153 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:13.902851 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f6ec8b-9207-4ec3-b153-5ae82037e822 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:13.910734 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 251/507] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Sep 9 20:47:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:13.913952 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6ec8b-9207-4ec3-b153-5ae82037e822 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:13.922011 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:13.944617 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04429174-4f52-4206-b3d3-4496663d48a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:13.944617 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04429174-4f52-4206-b3d3-4496663d48a2 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:14.011602 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6ec8b-9207-4ec3-b153-5ae82037e822 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:14.038062 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 257/508] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:47:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:47:14.039408 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04429174-4f52-4206-b3d3-4496663d48a2 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:14.050668 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0199de-0ce5-45d5-9de4-b3db2b17775f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:14.051296 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0199de-0ce5-45d5-9de4-b3db2b17775f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:14.051577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0199de-0ce5-45d5-9de4-b3db2b17775f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:14.051850 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0199de-0ce5-45d5-9de4-b3db2b17775f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:14.063728 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0199de-0ce5-45d5-9de4-b3db2b17775f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:14.065210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0199de-0ce5-45d5-9de4-b3db2b17775f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:14.080637 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-ff0199de-0ce5-45d5-9de4-b3db2b17775f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:14.080637 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:14.080637 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:14.080637 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:14.080637 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:14.080637 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:14.080637 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:14.080637 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:14.080637 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:14.080637 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:14.080637 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:14.080637 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:14.080637 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:14.080637 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:14.080637 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:14.080637 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:14.081663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:14.081663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:14.081663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:14.081663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:14.081663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:14.081663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:14.081663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:14.081663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:14.081663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:14.081663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:14.081663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:14.081663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:14.081663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:14.081663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:14.081663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:14.081663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:14.081663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:14.082610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:14.082610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:14.082610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:14.082610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 252/509] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:14.082894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0199de-0ce5-45d5-9de4-b3db2b17775f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:14.123284 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:14.148324 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:14.158021 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04429174-4f52-4206-b3d3-4496663d48a2 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:14.188802 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0199de-0ce5-45d5-9de4-b3db2b17775f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:14.191836 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092737a7-95a3-400e-9259-59e4239e9dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:14.192472 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092737a7-95a3-400e-9259-59e4239e9dcb None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:14.192751 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092737a7-95a3-400e-9259-59e4239e9dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:14.193021 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092737a7-95a3-400e-9259-59e4239e9dcb None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:14.193745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092737a7-95a3-400e-9259-59e4239e9dcb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:14.194048 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092737a7-95a3-400e-9259-59e4239e9dcb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:14.195525 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092737a7-95a3-400e-9259-59e4239e9dcb None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:14.213049 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 258/510] 10.222.0.48 () {56 vars in 1133 bytes} [Fri Sep 9 20:47:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:14.213871 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092737a7-95a3-400e-9259-59e4239e9dcb None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:14.222618 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9d4e2a-f63e-4801-888a-30b0ff0c49ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:14.223359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d4e2a-f63e-4801-888a-30b0ff0c49ec None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:14.223637 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d4e2a-f63e-4801-888a-30b0ff0c49ec None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:14.223908 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d4e2a-f63e-4801-888a-30b0ff0c49ec None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:14.231880 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9d4e2a-f63e-4801-888a-30b0ff0c49ec None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:14.233350 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9d4e2a-f63e-4801-888a-30b0ff0c49ec None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:14.245408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-0f9d4e2a-f63e-4801-888a-30b0ff0c49ec None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 20:47:14.245408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:14.245408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:14.245408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:14.245408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:14.245408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:14.245408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:14.245408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:14.245408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:14.245408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:14.245408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:14.245408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:14.245408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:47:14.245408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:47:14.245408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:47:14.245408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:47:14.246327 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:14.246327 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:14.246327 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:47:14.246327 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:14.246327 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:47:14.246327 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:14.246327 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:14.246327 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:14.246327 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:14.246327 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:14.246327 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:14.246327 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:14.246327 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:14.246327 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:14.246327 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:14.246327 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:14.246327 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:14.246327 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:14.247220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:14.247220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:14.247220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:47:14.247220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:47:14.247220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:47:14.247220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:47:14.247220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:47:14.247220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:47:14.247220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 20:47:14.247220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:14.247220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 253/511] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:47:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:14.247220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9d4e2a-f63e-4801-888a-30b0ff0c49ec None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:14.316353 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092737a7-95a3-400e-9259-59e4239e9dcb None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:14.338331 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:14.347693 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9d4e2a-f63e-4801-888a-30b0ff0c49ec None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:14.350621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4ed8e-c8e0-4819-af73-409acf9cd39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:14.350621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4ed8e-c8e0-4819-af73-409acf9cd39f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:14.350621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4ed8e-c8e0-4819-af73-409acf9cd39f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:14.351502 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4ed8e-c8e0-4819-af73-409acf9cd39f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:14.361043 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4ed8e-c8e0-4819-af73-409acf9cd39f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:14.362486 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4ed8e-c8e0-4819-af73-409acf9cd39f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:14.364578 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:14.377754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-1be4ed8e-c8e0-4819-af73-409acf9cd39f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:14.377754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:14.377754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:14.377754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:14.377754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:14.377754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:14.377754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:14.377754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:14.377754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:14.377754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:14.377754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:14.377754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:14.377754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:14.377754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:14.377754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:14.377754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:14.378736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:14.378736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:14.378736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:14.378736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:14.378736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:14.378736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:14.378736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:14.378736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:14.378736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:14.378736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:14.378736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:14.378736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:14.378736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:14.378736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:14.378736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:14.378736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:14.378736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:14.379566 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:14.379566 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:14.379566 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:14.379566 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 259/512] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:14.379566 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4ed8e-c8e0-4819-af73-409acf9cd39f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:14.385573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4721239-2b90-4013-9803-2b9981554419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:14.386213 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4721239-2b90-4013-9803-2b9981554419 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:14.386553 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4721239-2b90-4013-9803-2b9981554419 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:14.390763 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4721239-2b90-4013-9803-2b9981554419 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:14.397412 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4721239-2b90-4013-9803-2b9981554419 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:14.399405 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4721239-2b90-4013-9803-2b9981554419 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:14.414217 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-c4721239-2b90-4013-9803-2b9981554419 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:14.414217 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:14.414217 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:14.414217 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:14.414217 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:14.414217 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:14.414217 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:14.414217 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:14.414217 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:14.414217 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:14.414217 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:14.414217 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:14.414217 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:14.414217 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:14.414217 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:14.414217 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:14.415912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:14.415912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:14.415912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:14.415912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:14.415912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:14.415912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:14.415912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:14.415912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:14.415912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:14.415912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:14.415912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:14.415912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:14.415912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:14.415912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:14.415912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:14.415912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:14.415912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:14.417790 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:14.417790 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:14.417790 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:14.417790 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 254/513] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:14.417790 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4721239-2b90-4013-9803-2b9981554419 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:14.493190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4ed8e-c8e0-4819-af73-409acf9cd39f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:14.517740 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4721239-2b90-4013-9803-2b9981554419 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:14.526367 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2fa93-49af-498f-be07-abf553eaf986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:14.526970 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2fa93-49af-498f-be07-abf553eaf986 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:14.527248 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2fa93-49af-498f-be07-abf553eaf986 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:14.527520 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2fa93-49af-498f-be07-abf553eaf986 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:14.528189 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2fa93-49af-498f-be07-abf553eaf986 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:14.528493 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2fa93-49af-498f-be07-abf553eaf986 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:14.529851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2fa93-49af-498f-be07-abf553eaf986 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:14.541495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 260/514] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:14.542519 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2fa93-49af-498f-be07-abf553eaf986 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:14.550980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e7d40-aa7e-4795-8d67-068daa23a33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:14.551657 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e7d40-aa7e-4795-8d67-068daa23a33f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:14.551964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e7d40-aa7e-4795-8d67-068daa23a33f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:14.552247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e7d40-aa7e-4795-8d67-068daa23a33f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:14.577052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:14.591812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:14.620541 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e7d40-aa7e-4795-8d67-068daa23a33f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:14.621999 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e7d40-aa7e-4795-8d67-068daa23a33f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:14.643156 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2fa93-49af-498f-be07-abf553eaf986 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:14.676736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad2120-afbf-4f9f-bdfb-63a7906eeccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:14.677816 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad2120-afbf-4f9f-bdfb-63a7906eeccc None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:14.678267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad2120-afbf-4f9f-bdfb-63a7906eeccc None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:14.678532 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad2120-afbf-4f9f-bdfb-63a7906eeccc None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:14.679928 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dad2120-afbf-4f9f-bdfb-63a7906eeccc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:14.680572 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dad2120-afbf-4f9f-bdfb-63a7906eeccc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:14.681995 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dad2120-afbf-4f9f-bdfb-63a7906eeccc None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:14.699049 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 261/515] 10.222.0.48 () {56 vars in 1137 bytes} [Fri Sep 9 20:47:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:14.699833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad2120-afbf-4f9f-bdfb-63a7906eeccc None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:14.722011 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 255/516] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:47:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:47:14.722910 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e7d40-aa7e-4795-8d67-068daa23a33f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:14.787593 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:14.791050 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:14.796353 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad2120-afbf-4f9f-bdfb-63a7906eeccc None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:14.820245 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e7d40-aa7e-4795-8d67-068daa23a33f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:14.828698 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5145b31-d64a-4c0f-b22b-6722236d3acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:14.829297 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5145b31-d64a-4c0f-b22b-6722236d3acd None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:14.829575 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5145b31-d64a-4c0f-b22b-6722236d3acd None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:14.829846 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5145b31-d64a-4c0f-b22b-6722236d3acd None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:14.830487 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5145b31-d64a-4c0f-b22b-6722236d3acd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:14.830788 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5145b31-d64a-4c0f-b22b-6722236d3acd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:14.832190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5145b31-d64a-4c0f-b22b-6722236d3acd None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:14.843897 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 262/517] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:14.844711 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5145b31-d64a-4c0f-b22b-6722236d3acd None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:14.853047 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cafec6e-a2ec-4dad-adf5-d24b1323be03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:14.853652 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cafec6e-a2ec-4dad-adf5-d24b1323be03 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:14.853930 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cafec6e-a2ec-4dad-adf5-d24b1323be03 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:14.854200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cafec6e-a2ec-4dad-adf5-d24b1323be03 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:14.854840 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cafec6e-a2ec-4dad-adf5-d24b1323be03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:14.855142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cafec6e-a2ec-4dad-adf5-d24b1323be03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:14.856642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cafec6e-a2ec-4dad-adf5-d24b1323be03 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:14.868638 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 256/518] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:14.869830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cafec6e-a2ec-4dad-adf5-d24b1323be03 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:14.946535 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5145b31-d64a-4c0f-b22b-6722236d3acd None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:14.967423 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cafec6e-a2ec-4dad-adf5-d24b1323be03 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:14.979535 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c0f0d-8db6-4b3c-9dfc-c5fae356d2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:14.980168 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c0f0d-8db6-4b3c-9dfc-c5fae356d2e3 None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:14.980447 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c0f0d-8db6-4b3c-9dfc-c5fae356d2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:14.980717 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c0f0d-8db6-4b3c-9dfc-c5fae356d2e3 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:14.981355 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6c0f0d-8db6-4b3c-9dfc-c5fae356d2e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:14.982787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6c0f0d-8db6-4b3c-9dfc-c5fae356d2e3 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:14.999440 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:15.005079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51446186-b435-4df0-a227-4bfd7c9ae40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:15.005681 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51446186-b435-4df0-a227-4bfd7c9ae40c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:15.005960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51446186-b435-4df0-a227-4bfd7c9ae40c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:15.006237 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51446186-b435-4df0-a227-4bfd7c9ae40c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:15.070681 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51446186-b435-4df0-a227-4bfd7c9ae40c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:15.072104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51446186-b435-4df0-a227-4bfd7c9ae40c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:15.090944 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 263/519] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Sep 9 20:47:14 2022] POST /identity/v3/users => generated 247 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:47:15.091838 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c0f0d-8db6-4b3c-9dfc-c5fae356d2e3 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:15.105200 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:15.182345 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 257/520] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:47:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:47:15.183540 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51446186-b435-4df0-a227-4bfd7c9ae40c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:15.214317 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c0f0d-8db6-4b3c-9dfc-c5fae356d2e3 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:15.218498 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:15.251788 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5907b79-cf3e-405e-9bbb-dee72020604d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:15.252468 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5907b79-cf3e-405e-9bbb-dee72020604d None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:15.252748 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5907b79-cf3e-405e-9bbb-dee72020604d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:15.253008 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5907b79-cf3e-405e-9bbb-dee72020604d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:15.265385 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5907b79-cf3e-405e-9bbb-dee72020604d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:15.266861 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5907b79-cf3e-405e-9bbb-dee72020604d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:15.281795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-e5907b79-cf3e-405e-9bbb-dee72020604d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:15.281795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:15.281795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:15.281795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:15.281795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:15.281795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:15.281795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:15.281795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:15.281795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:15.281795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:15.281795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:15.281795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:15.281795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:15.281795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:15.281795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:15.281795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:15.282714 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:15.282714 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:15.282714 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:15.282714 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:15.282714 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:15.282714 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:15.282714 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:15.282714 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:15.282714 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:15.282714 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:15.282714 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:15.282714 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:15.282714 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:15.282714 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:15.282714 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:15.282714 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:15.282714 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:15.283691 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:15.283691 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:15.283691 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:15.283691 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 264/521] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:15.283691 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5907b79-cf3e-405e-9bbb-dee72020604d None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:15.302369 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51446186-b435-4df0-a227-4bfd7c9ae40c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:15.340375 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755a212-e24d-4992-99d2-3e09343bf461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:15.340929 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755a212-e24d-4992-99d2-3e09343bf461 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:15.341206 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755a212-e24d-4992-99d2-3e09343bf461 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:15.341477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755a212-e24d-4992-99d2-3e09343bf461 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:15.349291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:15.349899 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a755a212-e24d-4992-99d2-3e09343bf461 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:15.351591 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a755a212-e24d-4992-99d2-3e09343bf461 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:15.363949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-a755a212-e24d-4992-99d2-3e09343bf461 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 20:47:15.363949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:15.363949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:15.363949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:15.363949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:15.363949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:15.363949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:15.363949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:15.363949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:15.363949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:15.363949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:15.363949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:15.363949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:47:15.363949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:47:15.363949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:47:15.363949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:47:15.365158 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:15.365158 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:15.365158 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:47:15.365158 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:15.365158 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:47:15.365158 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:15.365158 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:15.365158 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:15.365158 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:15.365158 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:15.365158 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:15.365158 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:15.365158 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:15.365158 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:15.365158 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:15.365158 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:15.365158 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:15.365158 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:15.366236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:15.366236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:15.366236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:47:15.366236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:47:15.366236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:47:15.366236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:47:15.366236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:47:15.366236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:47:15.366236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 20:47:15.366236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:15.368441 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 258/522] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Sep 9 20:47:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:15.369354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755a212-e24d-4992-99d2-3e09343bf461 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:15.396543 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5907b79-cf3e-405e-9bbb-dee72020604d None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:15.432326 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8101d30-d795-4107-a146-4a59c37c4a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:15.433121 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8101d30-d795-4107-a146-4a59c37c4a13 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:15.433404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8101d30-d795-4107-a146-4a59c37c4a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:15.433680 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8101d30-d795-4107-a146-4a59c37c4a13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:15.441202 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8101d30-d795-4107-a146-4a59c37c4a13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:15.442743 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8101d30-d795-4107-a146-4a59c37c4a13 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:15.455430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-f8101d30-d795-4107-a146-4a59c37c4a13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:15.455430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:15.455430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:15.455430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:15.455430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:15.455430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:15.455430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:15.455430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:15.455430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:15.455430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:15.455430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:15.455430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:15.455430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:47:15.455430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:47:15.455430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:47:15.456508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:47:15.456508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:15.456508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:15.456508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:15.456508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:15.456508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:15.456508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:15.456508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:15.456508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:15.456508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:15.456508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:15.456508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:15.456508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:15.456508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:15.456508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:15.456508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:15.456508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:15.457632 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:47:15.457632 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:47:15.457632 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:47:15.457632 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:47:15.457632 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:47:15.457632 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:47:15.457632 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:15.457632 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:15.464102 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 265/523] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:47:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:15.464102 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8101d30-d795-4107-a146-4a59c37c4a13 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:15.475955 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:15.485933 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755a212-e24d-4992-99d2-3e09343bf461 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:15.522486 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f0e4b-681d-41c0-92ad-25d86cb3715d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:15.523730 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f0e4b-681d-41c0-92ad-25d86cb3715d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:15.524097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f0e4b-681d-41c0-92ad-25d86cb3715d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:15.524441 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f0e4b-681d-41c0-92ad-25d86cb3715d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:15.538309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f0e4b-681d-41c0-92ad-25d86cb3715d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:15.539792 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f0e4b-681d-41c0-92ad-25d86cb3715d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:15.554123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-f73f0e4b-681d-41c0-92ad-25d86cb3715d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:15.554123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:15.554123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:15.554123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:15.554123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:15.554123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:15.554123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:15.554123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:15.554123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:15.554123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:15.554123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:15.554123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:15.554123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:15.554123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:15.554123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:15.554123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:15.555093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:15.555093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:15.555093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:15.555093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:15.555093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:15.555093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:15.555093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:15.555093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:15.555093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:15.555093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:15.555093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:15.555093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:15.555093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:15.555093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:15.555093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:15.555093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:15.555093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:15.555947 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:15.555947 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:15.555947 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:15.555947 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 259/524] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:15.556199 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f0e4b-681d-41c0-92ad-25d86cb3715d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:15.576985 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8101d30-d795-4107-a146-4a59c37c4a13 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:15.600687 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:15.622934 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c3b0b-815a-42b2-9dc5-6aa4f8cbb92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:15.623548 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c3b0b-815a-42b2-9dc5-6aa4f8cbb92e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:15.623830 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c3b0b-815a-42b2-9dc5-6aa4f8cbb92e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:15.624138 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c3b0b-815a-42b2-9dc5-6aa4f8cbb92e None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:15.624793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c3b0b-815a-42b2-9dc5-6aa4f8cbb92e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:15.625099 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c3b0b-815a-42b2-9dc5-6aa4f8cbb92e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:15.626470 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c3b0b-815a-42b2-9dc5-6aa4f8cbb92e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:15.639953 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 266/525] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:15.639953 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c3b0b-815a-42b2-9dc5-6aa4f8cbb92e None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:15.670123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f0e4b-681d-41c0-92ad-25d86cb3715d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:15.707877 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f7b27-6214-4e6c-bb4f-e12e9a72a067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:15.708514 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f7b27-6214-4e6c-bb4f-e12e9a72a067 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:15.708792 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f7b27-6214-4e6c-bb4f-e12e9a72a067 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:15.709065 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f7b27-6214-4e6c-bb4f-e12e9a72a067 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:15.709795 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f7b27-6214-4e6c-bb4f-e12e9a72a067 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:15.710101 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f7b27-6214-4e6c-bb4f-e12e9a72a067 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:15.711562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f7b27-6214-4e6c-bb4f-e12e9a72a067 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:15.744485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:15.748101 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c3b0b-815a-42b2-9dc5-6aa4f8cbb92e None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:15.752872 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 260/526] 10.222.0.48 () {56 vars in 1139 bytes} [Fri Sep 9 20:47:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:15.752872 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f7b27-6214-4e6c-bb4f-e12e9a72a067 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:15.789253 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a492ea-7929-4389-994f-02fa9a94d31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:15.789858 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a492ea-7929-4389-994f-02fa9a94d31f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:15.790137 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a492ea-7929-4389-994f-02fa9a94d31f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:15.790410 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a492ea-7929-4389-994f-02fa9a94d31f None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:15.791101 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a492ea-7929-4389-994f-02fa9a94d31f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:15.791408 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a492ea-7929-4389-994f-02fa9a94d31f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:15.792882 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a492ea-7929-4389-994f-02fa9a94d31f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:15.811099 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 267/527] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:47:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:15.811923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a492ea-7929-4389-994f-02fa9a94d31f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:15.858635 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:15.863934 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f7b27-6214-4e6c-bb4f-e12e9a72a067 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:15.900417 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c76722-9791-4743-9e26-03564063f8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:15.901172 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c76722-9791-4743-9e26-03564063f8e6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:15.901841 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c76722-9791-4743-9e26-03564063f8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:15.901841 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c76722-9791-4743-9e26-03564063f8e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:15.902625 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c76722-9791-4743-9e26-03564063f8e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:15.903080 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c76722-9791-4743-9e26-03564063f8e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:15.904550 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c76722-9791-4743-9e26-03564063f8e6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:15.916436 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 261/528] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:15.917339 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c76722-9791-4743-9e26-03564063f8e6 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:15.925289 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a492ea-7929-4389-994f-02fa9a94d31f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:15.961666 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c22108-a29a-405b-a91b-dc39d1f13c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:15.962500 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c22108-a29a-405b-a91b-dc39d1f13c94 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:15.962901 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c22108-a29a-405b-a91b-dc39d1f13c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:15.963359 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c22108-a29a-405b-a91b-dc39d1f13c94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:15.970793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c22108-a29a-405b-a91b-dc39d1f13c94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:15.972509 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c22108-a29a-405b-a91b-dc39d1f13c94 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:15.981769 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:15.986313 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-62c22108-a29a-405b-a91b-dc39d1f13c94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:15.986313 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:15.986313 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:15.986313 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:15.986313 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:15.986313 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:15.986313 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:15.986313 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:15.986313 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:15.986313 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:15.986313 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:15.986313 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:15.986313 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:47:15.986313 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:47:15.986313 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:47:15.987312 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:47:15.987312 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:15.987312 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:15.987312 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:15.987312 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:15.987312 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:15.987312 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:15.987312 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:15.987312 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:15.987312 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:15.987312 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:15.987312 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:15.987312 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:15.987312 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:15.987312 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:15.987312 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:15.987312 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:15.995400 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:47:15.995400 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:47:15.995400 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:47:15.995400 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:47:15.995400 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:47:15.995400 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:47:15.995400 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:15.995400 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:15.995400 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 268/529] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:47:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:15.995400 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c22108-a29a-405b-a91b-dc39d1f13c94 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:16.036795 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c76722-9791-4743-9e26-03564063f8e6 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:16.072204 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ac6c7-116e-489b-9675-6a0913a9a369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:16.073118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ac6c7-116e-489b-9675-6a0913a9a369 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:16.073599 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ac6c7-116e-489b-9675-6a0913a9a369 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:16.074070 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ac6c7-116e-489b-9675-6a0913a9a369 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:16.099930 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:16.103414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c22108-a29a-405b-a91b-dc39d1f13c94 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:16.138622 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b0833-6323-42a0-90d8-d32cc168ea18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:16.139160 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b0833-6323-42a0-90d8-d32cc168ea18 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:16.139439 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b0833-6323-42a0-90d8-d32cc168ea18 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:16.139718 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b0833-6323-42a0-90d8-d32cc168ea18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:16.140801 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8b0833-6323-42a0-90d8-d32cc168ea18 None admin] RBAC: Adding query filter params (role.id=1e000597af164d54a40df2a39c729820, scope.project.id=91319aa782334fb9930770029b74e84a, user.id=7832e97d7e97439e821268d05816ae3d) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:16.141126 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8b0833-6323-42a0-90d8-d32cc168ea18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:16.143031 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8b0833-6323-42a0-90d8-d32cc168ea18 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:16.149650 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 269/530] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:47:15 2022] GET /identity/v3/role_assignments?user.id=7832e97d7e97439e821268d05816ae3d&scope.project.id=91319aa782334fb9930770029b74e84a&role.id=1e000597af164d54a40df2a39c729820 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:16.150084 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ac6c7-116e-489b-9675-6a0913a9a369 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:16.150754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b0833-6323-42a0-90d8-d32cc168ea18 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:16.151745 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ac6c7-116e-489b-9675-6a0913a9a369 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:16.217124 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:16.276242 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 262/531] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:47:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:47:16.277371 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ac6c7-116e-489b-9675-6a0913a9a369 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:16.293792 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b0833-6323-42a0-90d8-d32cc168ea18 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:16.335442 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d9b21-92d9-487a-a7a9-0cbf72e4466c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:16.336387 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d9b21-92d9-487a-a7a9-0cbf72e4466c None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:16.336809 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d9b21-92d9-487a-a7a9-0cbf72e4466c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:16.337238 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d9b21-92d9-487a-a7a9-0cbf72e4466c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:16.345204 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d9b21-92d9-487a-a7a9-0cbf72e4466c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:16.347006 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d9b21-92d9-487a-a7a9-0cbf72e4466c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:16.363817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-588d9b21-92d9-487a-a7a9-0cbf72e4466c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:16.363817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:16.363817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:16.363817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:16.363817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:16.363817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:16.363817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:16.363817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:16.363817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:16.363817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:16.363817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:16.363817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:16.363817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:47:16.363817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:47:16.363817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:47:16.364876 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:47:16.364876 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:16.364876 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:16.364876 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:16.364876 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:16.364876 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:16.364876 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:16.364876 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:16.364876 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:16.364876 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:16.364876 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:16.364876 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:16.364876 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:16.364876 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:16.364876 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:16.364876 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:16.364876 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:16.365906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:47:16.365906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:47:16.365906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:47:16.365906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:47:16.365906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:47:16.365906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:47:16.365906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:16.365906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:16.373746 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 270/532] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:47:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:16.376393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d9b21-92d9-487a-a7a9-0cbf72e4466c None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:16.409410 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ac6c7-116e-489b-9675-6a0913a9a369 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:16.410938 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:16.448962 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a909722-5bdb-42db-9987-295628166086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:16.449958 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a909722-5bdb-42db-9987-295628166086 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:16.450456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a909722-5bdb-42db-9987-295628166086 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:16.450924 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a909722-5bdb-42db-9987-295628166086 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:16.451971 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a909722-5bdb-42db-9987-295628166086 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:16.452542 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a909722-5bdb-42db-9987-295628166086 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:16.454254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a909722-5bdb-42db-9987-295628166086 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:16.473982 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 263/533] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:47:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:16.480522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a909722-5bdb-42db-9987-295628166086 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:16.497166 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d9b21-92d9-487a-a7a9-0cbf72e4466c None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:16.516556 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:16.537562 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37884058-3d8e-410c-aeb8-15de68153eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:16.538195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37884058-3d8e-410c-aeb8-15de68153eaa None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:16.539027 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37884058-3d8e-410c-aeb8-15de68153eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:16.539295 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37884058-3d8e-410c-aeb8-15de68153eaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:16.551820 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37884058-3d8e-410c-aeb8-15de68153eaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:16.554906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37884058-3d8e-410c-aeb8-15de68153eaa None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:16.578170 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-37884058-3d8e-410c-aeb8-15de68153eaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:16.578170 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:16.578170 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:16.578170 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:16.578170 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:16.578170 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:16.578170 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:16.578170 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:16.578170 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:16.578170 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:16.578170 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:16.578170 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:16.578170 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:16.578170 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:16.578170 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:16.578170 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:16.579871 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:16.579871 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:16.579871 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:16.579871 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:16.579871 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:16.579871 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:16.579871 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:16.579871 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:16.579871 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:16.579871 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:16.579871 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:16.579871 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:16.579871 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:16.579871 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:16.579871 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:16.579871 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:16.579871 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:16.581539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:16.581539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:16.581539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:16.581539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 271/534] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:16.581539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37884058-3d8e-410c-aeb8-15de68153eaa None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:16.630975 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a909722-5bdb-42db-9987-295628166086 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:16.672368 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:16.683742 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa4049-e2a1-489e-b4df-b035ddecce0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:16.684588 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa4049-e2a1-489e-b4df-b035ddecce0c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:16.686236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa4049-e2a1-489e-b4df-b035ddecce0c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:16.686236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa4049-e2a1-489e-b4df-b035ddecce0c None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:16.686236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa4049-e2a1-489e-b4df-b035ddecce0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:16.686807 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa4049-e2a1-489e-b4df-b035ddecce0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:16.688420 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa4049-e2a1-489e-b4df-b035ddecce0c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:16.700248 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37884058-3d8e-410c-aeb8-15de68153eaa None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:16.701762 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 264/535] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:47:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:16.703628 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa4049-e2a1-489e-b4df-b035ddecce0c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:16.742850 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7603d8b-678f-424a-ab1a-1b13b208697f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:16.743633 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7603d8b-678f-424a-ab1a-1b13b208697f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:16.744023 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7603d8b-678f-424a-ab1a-1b13b208697f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:16.744440 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7603d8b-678f-424a-ab1a-1b13b208697f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:16.745283 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7603d8b-678f-424a-ab1a-1b13b208697f None admin] RBAC: Adding query filter params (role.id=1e000597af164d54a40df2a39c729820, scope.project.id=91319aa782334fb9930770029b74e84a, user.id=613b845a9b4d432daf8d478fc3970708) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:16.746124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7603d8b-678f-424a-ab1a-1b13b208697f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:16.748193 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7603d8b-678f-424a-ab1a-1b13b208697f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:16.755413 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 272/536] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:47:16 2022] GET /identity/v3/role_assignments?user.id=613b845a9b4d432daf8d478fc3970708&scope.project.id=91319aa782334fb9930770029b74e84a&role.id=1e000597af164d54a40df2a39c729820 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:16.756254 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7603d8b-678f-424a-ab1a-1b13b208697f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:16.820926 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:16.862835 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa4049-e2a1-489e-b4df-b035ddecce0c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:16.943006 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a31f8-02ac-4445-b230-4ad1c37d88f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:16.943544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a31f8-02ac-4445-b230-4ad1c37d88f3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:16.943825 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a31f8-02ac-4445-b230-4ad1c37d88f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:16.944130 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a31f8-02ac-4445-b230-4ad1c37d88f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:16.945340 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835a31f8-02ac-4445-b230-4ad1c37d88f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:16.945648 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835a31f8-02ac-4445-b230-4ad1c37d88f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:16.947093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835a31f8-02ac-4445-b230-4ad1c37d88f3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:16.962915 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7603d8b-678f-424a-ab1a-1b13b208697f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:16.979326 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 265/537] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:16.989095 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a31f8-02ac-4445-b230-4ad1c37d88f3 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:17.020095 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d4354-5339-4333-8b4b-3d6aa558d2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:17.020270 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d4354-5339-4333-8b4b-3d6aa558d2d4 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:17.020542 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d4354-5339-4333-8b4b-3d6aa558d2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:17.020817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d4354-5339-4333-8b4b-3d6aa558d2d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:17.021558 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d4354-5339-4333-8b4b-3d6aa558d2d4 None admin] RBAC: Adding query filter params (role.id=1e000597af164d54a40df2a39c729820, scope.project.id=91319aa782334fb9930770029b74e84a, user.id=cf91131ee5bb481bbb47097cad0c003a) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:17.022011 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d4354-5339-4333-8b4b-3d6aa558d2d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:17.025803 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d4354-5339-4333-8b4b-3d6aa558d2d4 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:17.032299 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 273/538] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:47:16 2022] GET /identity/v3/role_assignments?user.id=cf91131ee5bb481bbb47097cad0c003a&scope.project.id=91319aa782334fb9930770029b74e84a&role.id=1e000597af164d54a40df2a39c729820 => generated 259 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:17.170711 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a31f8-02ac-4445-b230-4ad1c37d88f3 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:17.222288 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393fac1-8c4a-44d1-8a8b-58532b4cd919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:17.222851 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393fac1-8c4a-44d1-8a8b-58532b4cd919 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:17.223126 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393fac1-8c4a-44d1-8a8b-58532b4cd919 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:17.223454 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393fac1-8c4a-44d1-8a8b-58532b4cd919 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:17.236671 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d393fac1-8c4a-44d1-8a8b-58532b4cd919 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:17.238146 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d393fac1-8c4a-44d1-8a8b-58532b4cd919 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:17.241082 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:17.255062 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-d393fac1-8c4a-44d1-8a8b-58532b4cd919 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:17.255062 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:17.255062 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:17.255062 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:17.255062 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:17.255062 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:17.255062 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:17.255062 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:17.255062 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:17.255062 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:17.255062 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:17.255062 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:17.255062 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:47:17.255062 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:47:17.255062 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:47:17.255992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:47:17.255992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:17.255992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:17.255992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:17.255992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:17.255992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:17.255992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:17.255992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:17.255992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:17.255992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:17.255992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:17.255992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:17.255992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:17.255992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:17.255992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:17.255992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:17.255992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:17.256899 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:47:17.256899 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:47:17.256899 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:47:17.256899 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:47:17.256899 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:47:17.256899 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:47:17.256899 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:17.256899 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:17.256899 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 266/539] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:47:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:47:17.263778 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d4354-5339-4333-8b4b-3d6aa558d2d4 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:17.284371 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:17.381697 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c755c3-439d-4bac-9476-45715c2d3c77 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:17.382048 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c755c3-439d-4bac-9476-45715c2d3c77 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:17.382275 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c755c3-439d-4bac-9476-45715c2d3c77 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:17.383736 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 267/540] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:47:17.389161 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3e28b-5e84-471c-b91d-45d06aba1bbc None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:17.389694 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3e28b-5e84-471c-b91d-45d06aba1bbc None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:17.390173 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3e28b-5e84-471c-b91d-45d06aba1bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:17.410699 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d9a3e28b-5e84-471c-b91d-45d06aba1bbc None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:17.412365 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d9a3e28b-5e84-471c-b91d-45d06aba1bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:17.439608 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d4354-5339-4333-8b4b-3d6aa558d2d4 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:17.506721 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fd3e92-3e1d-425e-a9c5-da37b26d2d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:17.507795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd3e92-3e1d-425e-a9c5-da37b26d2d5c None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:17.508192 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd3e92-3e1d-425e-a9c5-da37b26d2d5c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:17.508389 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd3e92-3e1d-425e-a9c5-da37b26d2d5c None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:17.508792 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 268/541] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:47:17.512311 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fd3e92-3e1d-425e-a9c5-da37b26d2d5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:17.512311 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fd3e92-3e1d-425e-a9c5-da37b26d2d5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:17.512311 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fd3e92-3e1d-425e-a9c5-da37b26d2d5c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:17.536768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 274/542] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:47:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:17.542614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a3e28b-5e84-471c-b91d-45d06aba1bbc None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:17.707912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a3e28b-5e84-471c-b91d-45d06aba1bbc None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:17.741866 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:17.748142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b7ecb8-ad5a-4f1b-af42-a2f9a543643f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:17.748755 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7ecb8-ad5a-4f1b-af42-a2f9a543643f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:17.749036 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7ecb8-ad5a-4f1b-af42-a2f9a543643f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:17.749312 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7ecb8-ad5a-4f1b-af42-a2f9a543643f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:17.750116 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b7ecb8-ad5a-4f1b-af42-a2f9a543643f None admin] RBAC: Adding query filter params (role.id=1e000597af164d54a40df2a39c729820, scope.project.id=91319aa782334fb9930770029b74e84a, user.id=8f5dfac06ac9492a86503ba133c638a9) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:17.750425 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b7ecb8-ad5a-4f1b-af42-a2f9a543643f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:17.751875 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b7ecb8-ad5a-4f1b-af42-a2f9a543643f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:17.758609 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 269/543] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:47:17 2022] GET /identity/v3/role_assignments?user.id=8f5dfac06ac9492a86503ba133c638a9&scope.project.id=91319aa782334fb9930770029b74e84a&role.id=1e000597af164d54a40df2a39c729820 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:17.865909 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fd3e92-3e1d-425e-a9c5-da37b26d2d5c None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:17.957516 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:18.005271 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fd3e92-3e1d-425e-a9c5-da37b26d2d5c None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:18.045410 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334d4be-74e1-4bc9-a0cf-6aa281190b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:18.046061 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334d4be-74e1-4bc9-a0cf-6aa281190b47 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:18.046342 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334d4be-74e1-4bc9-a0cf-6aa281190b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:18.046676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334d4be-74e1-4bc9-a0cf-6aa281190b47 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:18.064713 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7334d4be-74e1-4bc9-a0cf-6aa281190b47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:47:18.070332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-7334d4be-74e1-4bc9-a0cf-6aa281190b47 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:18.070332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:18.070332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:18.070332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:18.070332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:18.070332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:18.070332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:18.070332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:18.070332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:18.070332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:18.070332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:18.070332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:18.070332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:47:18.070332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:47:18.070332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:47:18.070332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:47:18.071188 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:47:18.071188 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:47:18.071188 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:47:18.071188 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:47:18.071188 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:18.071188 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:18.071188 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:18.071188 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:18.071188 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:18.071188 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:18.071188 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:18.071188 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:18.071188 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:18.071188 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:18.071188 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:18.071188 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:18.072393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:18.072393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:18.072393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:18.072393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:18.072393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:47:18.072393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:47:18.072393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:47:18.072393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:47:18.072393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:47:18.072393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:47:18.072393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:18.072393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:18.072393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 275/544] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Sep 9 20:47:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:18.079051 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b7ecb8-ad5a-4f1b-af42-a2f9a543643f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:18.223924 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b7ecb8-ad5a-4f1b-af42-a2f9a543643f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:18.272154 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa40d618-5bfd-47b5-8739-8a339c9f06be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:18.272735 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40d618-5bfd-47b5-8739-8a339c9f06be None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:18.273017 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40d618-5bfd-47b5-8739-8a339c9f06be None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:18.273290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40d618-5bfd-47b5-8739-8a339c9f06be None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:18.273962 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa40d618-5bfd-47b5-8739-8a339c9f06be None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:18.274269 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa40d618-5bfd-47b5-8739-8a339c9f06be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:18.275650 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa40d618-5bfd-47b5-8739-8a339c9f06be None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:18.287274 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 270/545] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Sep 9 20:47:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:18.293600 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334d4be-74e1-4bc9-a0cf-6aa281190b47 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:18.436766 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334d4be-74e1-4bc9-a0cf-6aa281190b47 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:18.481612 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:18.492136 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc76f0f-1a7f-4630-b66e-ff48052809bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:18.492788 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc76f0f-1a7f-4630-b66e-ff48052809bc None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:18.501808 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc76f0f-1a7f-4630-b66e-ff48052809bc None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:18.501808 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc76f0f-1a7f-4630-b66e-ff48052809bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:18.503875 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc76f0f-1a7f-4630-b66e-ff48052809bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:18.505388 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc76f0f-1a7f-4630-b66e-ff48052809bc None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:18.528063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-4fc76f0f-1a7f-4630-b66e-ff48052809bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:18.528063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:18.528063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:18.528063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:18.528063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:18.528063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:18.528063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:18.528063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:18.528063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:18.528063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:18.528063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:18.528063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:18.528063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:18.528063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:18.528063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:18.528063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:18.529140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:18.529140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:18.529140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:18.529140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:18.529140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:18.529140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:18.529140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:18.529140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:18.529140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:18.529140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:18.529140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:18.529140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:18.529140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:18.529140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:18.529140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:18.529140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:18.529140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:18.530302 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:18.530302 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:18.530302 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:18.530302 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 276/546] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:18.537265 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa40d618-5bfd-47b5-8739-8a339c9f06be None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:18.613194 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:18.668365 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa40d618-5bfd-47b5-8739-8a339c9f06be None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:18.689719 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c4dde-e07f-46f3-930c-c0a02808de11 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:18.690020 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c4dde-e07f-46f3-930c-c0a02808de11 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:18.690605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c4dde-e07f-46f3-930c-c0a02808de11 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:18.693829 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 277/547] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:47:18.697467 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631ba61-f140-420d-b51e-1b76bbe7232a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:18.697751 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631ba61-f140-420d-b51e-1b76bbe7232a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:18.698026 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631ba61-f140-420d-b51e-1b76bbe7232a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:18.718755 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-6631ba61-f140-420d-b51e-1b76bbe7232a None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:18.719975 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-6631ba61-f140-420d-b51e-1b76bbe7232a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:18.722623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c46643-800d-4166-8117-68343840828b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:18.723247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c46643-800d-4166-8117-68343840828b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:18.723533 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c46643-800d-4166-8117-68343840828b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:18.723808 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c46643-800d-4166-8117-68343840828b None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:18.724488 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c46643-800d-4166-8117-68343840828b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:18.724799 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c46643-800d-4166-8117-68343840828b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:18.726748 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c46643-800d-4166-8117-68343840828b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:18.752781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 271/548] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:18.760137 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c46643-800d-4166-8117-68343840828b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:18.847025 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 278/549] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:47:18.893111 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c46643-800d-4166-8117-68343840828b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:18.910125 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:18.928940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f79d1-406e-430a-9ac2-9c05046358ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:18.929661 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f79d1-406e-430a-9ac2-9c05046358ee None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:18.929943 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f79d1-406e-430a-9ac2-9c05046358ee None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:18.930210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f79d1-406e-430a-9ac2-9c05046358ee None admin] PATH_INFO: `/v3/users/swift` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:18.937586 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6f79d1-406e-430a-9ac2-9c05046358ee None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:18.939311 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6f79d1-406e-430a-9ac2-9c05046358ee None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:18.951472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-3e6f79d1-406e-430a-9ac2-9c05046358ee None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 20:47:18.951472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:18.951472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:18.951472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:18.951472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:18.951472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:18.951472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:18.951472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:18.951472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:18.951472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:18.951472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:18.951472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:18.951472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:47:18.951472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:47:18.951472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:47:18.951472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:47:18.952527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:18.952527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:18.952527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:47:18.952527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:18.952527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:47:18.952527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:18.952527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:18.952527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:18.952527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:18.952527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:18.952527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:18.952527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:18.952527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:18.952527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:18.952527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:18.952527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:18.952527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:18.952527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:18.953459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:18.953459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:18.953459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:47:18.953459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:47:18.953459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:47:18.953459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:47:18.953459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:47:18.953459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:47:18.953459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 20:47:18.953459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:18.957497 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 272/550] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:47:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:47:18.963785 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631ba61-f140-420d-b51e-1b76bbe7232a None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:19.090383 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631ba61-f140-420d-b51e-1b76bbe7232a None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:19.100365 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:19.134313 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc3d78-ff3b-4017-a7ca-ac69e77c6890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:19.135310 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc3d78-ff3b-4017-a7ca-ac69e77c6890 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:19.135833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc3d78-ff3b-4017-a7ca-ac69e77c6890 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:19.136280 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc3d78-ff3b-4017-a7ca-ac69e77c6890 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:19.139339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecc3d78-ff3b-4017-a7ca-ac69e77c6890 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:19.139847 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecc3d78-ff3b-4017-a7ca-ac69e77c6890 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:19.141508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecc3d78-ff3b-4017-a7ca-ac69e77c6890 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:19.162385 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 279/551] 10.222.0.48 () {56 vars in 1135 bytes} [Fri Sep 9 20:47:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:19.168332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f79d1-406e-430a-9ac2-9c05046358ee None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:19.248883 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc3d78-ff3b-4017-a7ca-ac69e77c6890 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:19.329242 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f79d1-406e-430a-9ac2-9c05046358ee None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:19.396476 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad86b71-b261-48b6-91e6-16bc7f8fc78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:19.398490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad86b71-b261-48b6-91e6-16bc7f8fc78e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:19.399168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad86b71-b261-48b6-91e6-16bc7f8fc78e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:19.401175 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad86b71-b261-48b6-91e6-16bc7f8fc78e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:19.434825 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc3d78-ff3b-4017-a7ca-ac69e77c6890 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:19.484442 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:19.499161 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f285c-cd9d-4c05-a7d9-592752d04ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:19.500161 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f285c-cd9d-4c05-a7d9-592752d04ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:19.501739 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f285c-cd9d-4c05-a7d9-592752d04ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:19.502209 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f285c-cd9d-4c05-a7d9-592752d04ed8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:19.518252 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f285c-cd9d-4c05-a7d9-592752d04ed8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:47:19.523536 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-452f285c-cd9d-4c05-a7d9-592752d04ed8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:19.523536 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:19.523536 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:19.523536 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:19.523536 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:19.523536 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:19.523536 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:19.523536 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:19.523536 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:19.523536 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:19.523536 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:19.523536 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:19.523536 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:47:19.523536 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:47:19.523536 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:47:19.523536 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:47:19.524594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:47:19.524594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:47:19.524594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:47:19.524594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:47:19.524594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:19.524594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:19.524594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:19.524594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:19.524594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:19.524594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:19.524594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:19.524594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:19.524594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:19.524594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:19.524594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:19.524594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:19.525772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:19.525772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:19.525772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:19.525772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:19.525772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:47:19.525772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:47:19.525772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:47:19.525772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:47:19.525772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:47:19.525772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:47:19.525772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:19.525772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:19.525772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 280/552] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Sep 9 20:47:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:47:19.526638 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f105c-dfd9-446a-ac84-a6dcb13c4395 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:19.526638 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f105c-dfd9-446a-ac84-a6dcb13c4395 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:19.527009 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f105c-dfd9-446a-ac84-a6dcb13c4395 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:19.527674 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad86b71-b261-48b6-91e6-16bc7f8fc78e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:19.529785 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad86b71-b261-48b6-91e6-16bc7f8fc78e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:19.533559 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 281/553] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:19.534079 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f105c-dfd9-446a-ac84-a6dcb13c4395 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:19.562140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:19.662560 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 273/554] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:47:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:47:19.662560 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6c5da-ca9a-46a5-b16e-78860c9d9d85 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:19.662560 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6c5da-ca9a-46a5-b16e-78860c9d9d85 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:19.662560 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6c5da-ca9a-46a5-b16e-78860c9d9d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:19.684622 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-39f6c5da-ca9a-46a5-b16e-78860c9d9d85 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:19.686458 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-39f6c5da-ca9a-46a5-b16e-78860c9d9d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:19.701673 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f105c-dfd9-446a-ac84-a6dcb13c4395 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:19.752177 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2014027-9418-4894-ba44-23daf7fbe016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:19.753206 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2014027-9418-4894-ba44-23daf7fbe016 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:19.753712 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2014027-9418-4894-ba44-23daf7fbe016 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:19.754186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2014027-9418-4894-ba44-23daf7fbe016 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:19.755382 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2014027-9418-4894-ba44-23daf7fbe016 None admin] RBAC: Adding query filter params (name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:19.755936 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2014027-9418-4894-ba44-23daf7fbe016 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:19.759046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2014027-9418-4894-ba44-23daf7fbe016 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:19.768140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 274/555] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:47:19.768140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6c5da-ca9a-46a5-b16e-78860c9d9d85 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:19.776367 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 282/556] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Sep 9 20:47:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:19.780817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77750d45-7fcd-4421-8273-a0cac76c2fa2 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:19.781305 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77750d45-7fcd-4421-8273-a0cac76c2fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:19.781583 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77750d45-7fcd-4421-8273-a0cac76c2fa2 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:19.783020 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 283/557] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:19.786046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77750d45-7fcd-4421-8273-a0cac76c2fa2 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:19.786211 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:19.913057 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:19.931638 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77750d45-7fcd-4421-8273-a0cac76c2fa2 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:19.932641 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6c5da-ca9a-46a5-b16e-78860c9d9d85 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:19.971593 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a3f0bb-caef-41e9-957d-e604db7beb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:19.972570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3f0bb-caef-41e9-957d-e604db7beb6f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:19.972570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3f0bb-caef-41e9-957d-e604db7beb6f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:19.972823 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3f0bb-caef-41e9-957d-e604db7beb6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:19.984031 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a3f0bb-caef-41e9-957d-e604db7beb6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:19.985496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a3f0bb-caef-41e9-957d-e604db7beb6f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:20.002029 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-94a3f0bb-caef-41e9-957d-e604db7beb6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:20.002029 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:20.002029 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:20.002029 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:20.002029 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:20.002029 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:20.002029 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:20.002029 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:20.002029 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:20.002029 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:20.002029 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:20.002029 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:20.002029 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:20.002029 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:20.002029 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:20.002029 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:20.003787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:20.003787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:20.003787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:20.003787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:20.003787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:20.003787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:20.003787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:20.003787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:20.003787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:20.003787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:20.003787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:20.003787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:20.003787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:20.003787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:20.003787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:20.003787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:20.003787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:20.005594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:20.005594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:20.005594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:20.008526 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 284/558] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:20.009965 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158bda4d-b1a4-44c6-9d25-ac6aa6067a97 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:20.010249 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158bda4d-b1a4-44c6-9d25-ac6aa6067a97 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:20.010525 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158bda4d-b1a4-44c6-9d25-ac6aa6067a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:20.014760 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ac370-6b13-4f83-b132-bec59c1c37fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:20.016206 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ac370-6b13-4f83-b132-bec59c1c37fb None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:20.017011 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ac370-6b13-4f83-b132-bec59c1c37fb None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:20.017754 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ac370-6b13-4f83-b132-bec59c1c37fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:20.034374 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-158bda4d-b1a4-44c6-9d25-ac6aa6067a97 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:20.035575 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-158bda4d-b1a4-44c6-9d25-ac6aa6067a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:20.042289 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2ac370-6b13-4f83-b132-bec59c1c37fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:20.045924 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2ac370-6b13-4f83-b132-bec59c1c37fb None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:20.065662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-0b2ac370-6b13-4f83-b132-bec59c1c37fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:20.065662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:20.065662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:20.065662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:20.065662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:20.065662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:20.065662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:20.065662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:20.065662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:20.065662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:20.065662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:20.065662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:20.065662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:20.065662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:20.065662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:20.065662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:20.066644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:20.066644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:20.066644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:20.066644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:20.066644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:20.066644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:20.066644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:20.066644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:20.066644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:20.066644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:20.066644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:20.066644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:20.066644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:20.066644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:20.066644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:20.066644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:20.066644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:20.067534 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:20.067534 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:20.067534 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:20.067534 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 275/559] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:20.067534 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ac370-6b13-4f83-b132-bec59c1c37fb None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:20.124106 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 285/560] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:47:20.124106 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158bda4d-b1a4-44c6-9d25-ac6aa6067a97 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:20.138794 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:20.188694 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ac370-6b13-4f83-b132-bec59c1c37fb None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:20.207402 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:20.224750 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47f0f9-2459-47d0-8221-ce213c8eceb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:20.225364 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47f0f9-2459-47d0-8221-ce213c8eceb6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:20.225678 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47f0f9-2459-47d0-8221-ce213c8eceb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:20.225960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47f0f9-2459-47d0-8221-ce213c8eceb6 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:20.226689 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d47f0f9-2459-47d0-8221-ce213c8eceb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:20.227001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d47f0f9-2459-47d0-8221-ce213c8eceb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:20.228436 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d47f0f9-2459-47d0-8221-ce213c8eceb6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:20.246440 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 276/561] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:20.247319 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47f0f9-2459-47d0-8221-ce213c8eceb6 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:20.247528 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158bda4d-b1a4-44c6-9d25-ac6aa6067a97 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:20.291523 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ef3ab-b5c4-4a49-bd72-db8cb4d57db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:20.292181 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ef3ab-b5c4-4a49-bd72-db8cb4d57db9 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:20.292768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ef3ab-b5c4-4a49-bd72-db8cb4d57db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:20.293041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ef3ab-b5c4-4a49-bd72-db8cb4d57db9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:20.306590 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ef3ab-b5c4-4a49-bd72-db8cb4d57db9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:47:20.313485 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-7c2ef3ab-b5c4-4a49-bd72-db8cb4d57db9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:20.313485 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:20.313485 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:20.313485 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:20.313485 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:20.313485 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:20.313485 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:20.313485 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:20.313485 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:20.313485 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:20.313485 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:20.313485 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:20.313485 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:47:20.313485 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:47:20.313485 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:47:20.313485 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:47:20.314386 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:47:20.314386 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:47:20.314386 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:47:20.314386 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:47:20.314386 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:20.314386 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:20.314386 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:20.314386 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:20.314386 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:20.314386 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:20.314386 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:20.314386 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:20.314386 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:20.314386 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:20.314386 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:20.314386 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:20.315205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:20.315205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:20.315205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:20.315205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:20.315205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:47:20.315205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:47:20.315205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:47:20.315205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:47:20.315205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:47:20.315205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:47:20.315205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:20.315205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:20.316329 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 286/562] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Sep 9 20:47:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:20.320089 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ef3ab-b5c4-4a49-bd72-db8cb4d57db9 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:20.369348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47f0f9-2459-47d0-8221-ce213c8eceb6 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:20.393756 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:20.412134 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68e3e5-e5a5-4fe3-8500-4f87fa12dc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:20.412766 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68e3e5-e5a5-4fe3-8500-4f87fa12dc13 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:20.413048 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68e3e5-e5a5-4fe3-8500-4f87fa12dc13 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:20.413377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68e3e5-e5a5-4fe3-8500-4f87fa12dc13 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:20.414247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b68e3e5-e5a5-4fe3-8500-4f87fa12dc13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:20.414562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b68e3e5-e5a5-4fe3-8500-4f87fa12dc13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:20.416011 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b68e3e5-e5a5-4fe3-8500-4f87fa12dc13 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:20.434717 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 277/563] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:20.434717 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68e3e5-e5a5-4fe3-8500-4f87fa12dc13 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:20.441952 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ef3ab-b5c4-4a49-bd72-db8cb4d57db9 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:20.472585 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:20.481110 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f04a9e4-8d29-437c-aa7a-5180ad5a74cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:20.481734 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04a9e4-8d29-437c-aa7a-5180ad5a74cb None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:20.487568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04a9e4-8d29-437c-aa7a-5180ad5a74cb None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:20.487568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04a9e4-8d29-437c-aa7a-5180ad5a74cb None admin] PATH_INFO: `/v3/users/glance` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:20.489927 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f04a9e4-8d29-437c-aa7a-5180ad5a74cb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:20.491925 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f04a9e4-8d29-437c-aa7a-5180ad5a74cb None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:20.505668 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-1f04a9e4-8d29-437c-aa7a-5180ad5a74cb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 20:47:20.505668 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:20.505668 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:20.505668 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:20.505668 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:20.505668 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:20.505668 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:20.505668 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:20.505668 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:20.505668 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:20.505668 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:20.505668 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:20.505668 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:47:20.505668 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:47:20.505668 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:47:20.505668 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:47:20.506659 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:20.506659 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:20.506659 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:47:20.506659 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:20.506659 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:47:20.506659 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:20.506659 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:20.506659 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:20.506659 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:20.506659 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:20.506659 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:20.506659 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:20.506659 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:20.506659 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:20.506659 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:20.506659 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:20.506659 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:20.506659 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:20.512745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:20.512745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:20.512745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:47:20.512745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:47:20.512745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:47:20.512745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:47:20.512745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:47:20.512745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:47:20.512745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 20:47:20.512745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:20.512745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 287/564] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:47:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:20.512745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f04a9e4-8d29-437c-aa7a-5180ad5a74cb None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:20.539473 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68e3e5-e5a5-4fe3-8500-4f87fa12dc13 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:20.573117 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7b3be-27e5-44f0-b6f8-8808c691bf61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:20.573706 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7b3be-27e5-44f0-b6f8-8808c691bf61 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:20.573985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7b3be-27e5-44f0-b6f8-8808c691bf61 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:20.574251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7b3be-27e5-44f0-b6f8-8808c691bf61 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:20.574909 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae7b3be-27e5-44f0-b6f8-8808c691bf61 None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:20.575212 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae7b3be-27e5-44f0-b6f8-8808c691bf61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:20.576607 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae7b3be-27e5-44f0-b6f8-8808c691bf61 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:20.584345 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 278/565] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Sep 9 20:47:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:20.585146 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7b3be-27e5-44f0-b6f8-8808c691bf61 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:20.612474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f04a9e4-8d29-437c-aa7a-5180ad5a74cb None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:20.655140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:20.680230 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd412739-43c1-4baa-8095-ffe688f8fd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:20.680858 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd412739-43c1-4baa-8095-ffe688f8fd74 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:20.681137 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd412739-43c1-4baa-8095-ffe688f8fd74 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:20.681409 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd412739-43c1-4baa-8095-ffe688f8fd74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:20.691435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd412739-43c1-4baa-8095-ffe688f8fd74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:47:20.696706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-cd412739-43c1-4baa-8095-ffe688f8fd74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:20.696706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:20.696706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:20.696706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:20.696706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:20.696706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:20.696706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:20.696706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:20.696706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:20.696706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:20.696706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:20.696706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:20.696706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:47:20.696706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:47:20.696706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:47:20.696706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:47:20.697653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:47:20.697653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:47:20.697653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:47:20.697653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:47:20.697653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:20.697653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:20.697653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:20.697653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:20.697653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:20.697653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:20.697653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:20.697653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:20.697653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:20.697653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:20.697653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:20.697653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:20.698497 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:20.698497 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:20.698497 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:20.698497 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:20.698497 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:47:20.698497 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:47:20.698497 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:47:20.698497 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:47:20.698497 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:47:20.698497 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:47:20.698497 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:20.698497 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:20.698497 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 288/566] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Sep 9 20:47:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:20.698497 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd412739-43c1-4baa-8095-ffe688f8fd74 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:20.714384 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7b3be-27e5-44f0-b6f8-8808c691bf61 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:20.747721 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5cc5f8-b009-4a90-9b45-31f7cc84db48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:20.747900 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:20.748375 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cc5f8-b009-4a90-9b45-31f7cc84db48 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:20.748655 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cc5f8-b009-4a90-9b45-31f7cc84db48 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:20.748926 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cc5f8-b009-4a90-9b45-31f7cc84db48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:20.755349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5cc5f8-b009-4a90-9b45-31f7cc84db48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:20.756811 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5cc5f8-b009-4a90-9b45-31f7cc84db48 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:20.767734 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-1c5cc5f8-b009-4a90-9b45-31f7cc84db48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:20.767734 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:20.767734 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:20.767734 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:20.767734 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:20.767734 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:20.767734 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:20.767734 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:20.767734 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:20.767734 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:20.767734 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:20.767734 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:20.767734 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:47:20.767734 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:47:20.767734 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:47:20.768675 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:47:20.768675 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:20.768675 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:20.768675 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:20.768675 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:20.768675 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:20.768675 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:20.768675 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:20.768675 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:20.768675 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:20.768675 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:20.768675 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:20.768675 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:20.768675 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:20.768675 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:20.768675 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:20.768675 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:20.769562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:47:20.769562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:47:20.769562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:47:20.769562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:47:20.769562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:47:20.769562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:47:20.769562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:20.769562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:20.769562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 279/567] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:47:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:20.770453 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ecaa0-f388-4485-8695-0be2f6df1081 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:20.770453 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ecaa0-f388-4485-8695-0be2f6df1081 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:20.770626 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ecaa0-f388-4485-8695-0be2f6df1081 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:20.775876 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 280/568] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:20.775876 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ecaa0-f388-4485-8695-0be2f6df1081 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:20.799802 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd412739-43c1-4baa-8095-ffe688f8fd74 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:20.833365 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd341a-b09a-40c5-b7db-5bea472b258b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:20.833996 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd341a-b09a-40c5-b7db-5bea472b258b None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:20.834278 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd341a-b09a-40c5-b7db-5bea472b258b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:20.834548 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd341a-b09a-40c5-b7db-5bea472b258b None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:20.835300 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd341a-b09a-40c5-b7db-5bea472b258b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:20.835603 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd341a-b09a-40c5-b7db-5bea472b258b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:20.837077 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd341a-b09a-40c5-b7db-5bea472b258b None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:20.854061 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 289/569] 10.222.0.48 () {56 vars in 1137 bytes} [Fri Sep 9 20:47:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:20.854849 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd341a-b09a-40c5-b7db-5bea472b258b None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:20.874069 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:20.877926 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ecaa0-f388-4485-8695-0be2f6df1081 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:20.911107 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5bf3a5-f0fe-4d9f-ba61-efa075a895cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:20.911728 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bf3a5-f0fe-4d9f-ba61-efa075a895cb None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:20.912006 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bf3a5-f0fe-4d9f-ba61-efa075a895cb None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:20.912296 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bf3a5-f0fe-4d9f-ba61-efa075a895cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:20.922939 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5bf3a5-f0fe-4d9f-ba61-efa075a895cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:20.924404 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5bf3a5-f0fe-4d9f-ba61-efa075a895cb None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:20.939015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-fc5bf3a5-f0fe-4d9f-ba61-efa075a895cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:20.939015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:20.939015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:20.939015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:20.939015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:20.939015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:20.939015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:20.939015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:20.939015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:20.939015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:20.939015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:20.939015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:20.939015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:20.939015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:20.939015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:20.939015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:20.940277 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:20.940277 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:20.940277 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:20.940277 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:20.940277 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:20.940277 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:20.940277 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:20.940277 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:20.940277 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:20.940277 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:20.940277 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:20.940277 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:20.940277 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:20.940277 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:20.940277 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:20.940277 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:20.940277 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:20.941225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:20.941225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:20.941225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:20.941225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 281/570] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:20.941225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5bf3a5-f0fe-4d9f-ba61-efa075a895cb None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:20.956943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd341a-b09a-40c5-b7db-5bea472b258b None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:20.965732 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:20.993441 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878b3b3-d130-4461-8278-077de47cb8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:20.994059 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878b3b3-d130-4461-8278-077de47cb8ba None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:20.994307 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878b3b3-d130-4461-8278-077de47cb8ba None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:20.994575 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878b3b3-d130-4461-8278-077de47cb8ba None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:20.995226 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2878b3b3-d130-4461-8278-077de47cb8ba None admin] RBAC: Adding query filter params (name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:20.995525 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2878b3b3-d130-4461-8278-077de47cb8ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:20.997020 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2878b3b3-d130-4461-8278-077de47cb8ba None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:21.004560 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 290/571] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Sep 9 20:47:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:21.005686 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b3d38-12ba-4abf-964e-44849cb56bb4 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:21.005969 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b3d38-12ba-4abf-964e-44849cb56bb4 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:21.006237 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b3d38-12ba-4abf-964e-44849cb56bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:21.026057 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-b04b3d38-12ba-4abf-964e-44849cb56bb4 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:21.027234 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b04b3d38-12ba-4abf-964e-44849cb56bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:21.042679 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5bf3a5-f0fe-4d9f-ba61-efa075a895cb None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:21.074948 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5801af-1dd6-44ed-9008-27bcf81f5e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:21.075565 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5801af-1dd6-44ed-9008-27bcf81f5e2a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:21.075833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5801af-1dd6-44ed-9008-27bcf81f5e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:21.076126 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5801af-1dd6-44ed-9008-27bcf81f5e2a None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:21.076821 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5801af-1dd6-44ed-9008-27bcf81f5e2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:21.077132 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5801af-1dd6-44ed-9008-27bcf81f5e2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:21.078565 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5801af-1dd6-44ed-9008-27bcf81f5e2a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:21.090684 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 282/572] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:47:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:21.091562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5801af-1dd6-44ed-9008-27bcf81f5e2a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:21.092834 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 291/573] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:47:21.097007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b3d38-12ba-4abf-964e-44849cb56bb4 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:21.106405 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:21.194852 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5801af-1dd6-44ed-9008-27bcf81f5e2a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:21.195501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:21.199189 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b3d38-12ba-4abf-964e-44849cb56bb4 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:21.231430 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1dd776-a7c5-4d8e-af2f-b46726b9bd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:21.232360 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1dd776-a7c5-4d8e-af2f-b46726b9bd3a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:21.232844 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1dd776-a7c5-4d8e-af2f-b46726b9bd3a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:21.233256 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1dd776-a7c5-4d8e-af2f-b46726b9bd3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:21.234841 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbda7d4-9db0-4dbe-9fa7-d0ac0b1866ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:21.235454 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbda7d4-9db0-4dbe-9fa7-d0ac0b1866ed None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:21.235774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbda7d4-9db0-4dbe-9fa7-d0ac0b1866ed None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:21.236052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbda7d4-9db0-4dbe-9fa7-d0ac0b1866ed None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:21.236802 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbda7d4-9db0-4dbe-9fa7-d0ac0b1866ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:21.237115 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbda7d4-9db0-4dbe-9fa7-d0ac0b1866ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:21.238499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbda7d4-9db0-4dbe-9fa7-d0ac0b1866ed None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:21.245319 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1dd776-a7c5-4d8e-af2f-b46726b9bd3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:21.247150 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1dd776-a7c5-4d8e-af2f-b46726b9bd3a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:21.253496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 292/574] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:21.254335 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbda7d4-9db0-4dbe-9fa7-d0ac0b1866ed None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:21.268422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-fe1dd776-a7c5-4d8e-af2f-b46726b9bd3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:21.268422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:21.268422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:21.268422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:21.268422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:21.268422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:21.268422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:21.268422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:21.268422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:21.268422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:21.268422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:21.268422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:21.268422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:21.268422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:21.268422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:21.268422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:21.269446 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:21.269446 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:21.269446 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:21.269446 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:21.269446 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:21.269446 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:21.269446 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:21.269446 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:21.269446 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:21.269446 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:21.269446 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:21.269446 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:21.269446 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:21.269446 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:21.269446 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:21.269446 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:21.269446 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:21.270537 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:21.270537 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:21.270537 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:21.277002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 283/575] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:21.277988 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1dd776-a7c5-4d8e-af2f-b46726b9bd3a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:21.347726 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:21.363771 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbda7d4-9db0-4dbe-9fa7-d0ac0b1866ed None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:21.385387 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1dd776-a7c5-4d8e-af2f-b46726b9bd3a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:21.396899 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b92054-07f7-4ba1-8e2e-322c3beb09b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:21.397576 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b92054-07f7-4ba1-8e2e-322c3beb09b2 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:21.398366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b92054-07f7-4ba1-8e2e-322c3beb09b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:21.398640 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b92054-07f7-4ba1-8e2e-322c3beb09b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:21.408919 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b92054-07f7-4ba1-8e2e-322c3beb09b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:21.410343 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b92054-07f7-4ba1-8e2e-322c3beb09b2 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:21.418412 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af3980-83c8-4f32-94b1-7afea2a7d9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:21.419015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af3980-83c8-4f32-94b1-7afea2a7d9d3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:21.419294 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af3980-83c8-4f32-94b1-7afea2a7d9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:21.419568 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af3980-83c8-4f32-94b1-7afea2a7d9d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:21.420324 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0af3980-83c8-4f32-94b1-7afea2a7d9d3 None admin] RBAC: Adding query filter params (role.id=1e000597af164d54a40df2a39c729820, scope.project.id=91319aa782334fb9930770029b74e84a, user.id=7d28fa5d49d44120bd5eaec2d4e5fdbd) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:21.420632 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0af3980-83c8-4f32-94b1-7afea2a7d9d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:21.422044 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0af3980-83c8-4f32-94b1-7afea2a7d9d3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:21.424937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-14b92054-07f7-4ba1-8e2e-322c3beb09b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:21.424937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:21.424937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:21.424937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:21.424937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:21.424937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:21.424937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:21.424937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:21.424937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:21.424937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:21.424937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:21.424937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:21.424937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:21.424937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:21.424937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:21.424937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:21.425935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:21.425935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:21.425935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:21.425935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:21.425935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:21.425935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:21.425935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:21.425935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:21.425935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:21.425935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:21.425935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:21.425935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:21.425935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:21.425935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:21.425935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:21.425935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:21.425935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:21.426828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:21.426828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:21.426828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:21.426828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 293/576] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:21.426828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b92054-07f7-4ba1-8e2e-322c3beb09b2 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:21.431941 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 284/577] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:47:21 2022] GET /identity/v3/role_assignments?user.id=7d28fa5d49d44120bd5eaec2d4e5fdbd&scope.project.id=91319aa782334fb9930770029b74e84a&role.id=1e000597af164d54a40df2a39c729820 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:21.432761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af3980-83c8-4f32-94b1-7afea2a7d9d3 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:21.443526 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:21.546520 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b92054-07f7-4ba1-8e2e-322c3beb09b2 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:21.546735 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af3980-83c8-4f32-94b1-7afea2a7d9d3 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:21.576986 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:21.585004 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251fdfd-2794-4776-ba32-04c6fe94e1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:21.586534 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251fdfd-2794-4776-ba32-04c6fe94e1da None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:21.586682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0469aa-774e-4d31-86d8-01c058dfe132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:21.589224 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251fdfd-2794-4776-ba32-04c6fe94e1da None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:21.589224 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251fdfd-2794-4776-ba32-04c6fe94e1da None admin] PATH_INFO: `/v3/users/nova` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:21.589419 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0469aa-774e-4d31-86d8-01c058dfe132 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:21.589419 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0469aa-774e-4d31-86d8-01c058dfe132 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:21.589419 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0469aa-774e-4d31-86d8-01c058dfe132 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:21.589419 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0469aa-774e-4d31-86d8-01c058dfe132 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:21.589419 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0469aa-774e-4d31-86d8-01c058dfe132 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:21.590289 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0469aa-774e-4d31-86d8-01c058dfe132 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:21.597158 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251fdfd-2794-4776-ba32-04c6fe94e1da None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:21.598886 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251fdfd-2794-4776-ba32-04c6fe94e1da None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:21.608431 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 285/578] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:21.609373 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0469aa-774e-4d31-86d8-01c058dfe132 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:21.614487 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-4251fdfd-2794-4776-ba32-04c6fe94e1da None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:47:21.614487 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:21.614487 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:21.614487 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:21.614487 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:21.614487 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:21.614487 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:21.614487 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:21.614487 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:21.614487 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:21.614487 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:21.614487 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:21.614487 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:47:21.614487 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:47:21.614487 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:47:21.614487 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:47:21.618463 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:21.618463 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:21.618463 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:47:21.618463 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:21.618463 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:47:21.618463 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:21.618463 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:21.618463 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:21.618463 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:21.618463 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:21.618463 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:21.618463 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:21.618463 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:21.618463 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:21.618463 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:21.618463 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:21.618463 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:21.618463 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:21.619403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:21.619403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:21.619403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:47:21.619403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:47:21.619403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:47:21.619403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:47:21.619403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:47:21.619403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:47:21.619403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:47:21.619403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:21.619403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 294/579] 10.222.0.48 () {56 vars in 1093 bytes} [Fri Sep 9 20:47:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:21.619403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251fdfd-2794-4776-ba32-04c6fe94e1da None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:21.698440 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:21.728666 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0469aa-774e-4d31-86d8-01c058dfe132 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:21.731845 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251fdfd-2794-4776-ba32-04c6fe94e1da None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:21.764831 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e7903-3624-4066-a78f-3787aaf9d339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:21.765567 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e7903-3624-4066-a78f-3787aaf9d339 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:21.765843 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e7903-3624-4066-a78f-3787aaf9d339 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:21.766112 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e7903-3624-4066-a78f-3787aaf9d339 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:21.771655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5baeb8e-f776-4f51-af61-3a92680fc7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:21.772501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5baeb8e-f776-4f51-af61-3a92680fc7ed None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:21.772955 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5baeb8e-f776-4f51-af61-3a92680fc7ed None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:21.773225 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5baeb8e-f776-4f51-af61-3a92680fc7ed None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:21.774009 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5baeb8e-f776-4f51-af61-3a92680fc7ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:21.774166 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5baeb8e-f776-4f51-af61-3a92680fc7ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:21.775523 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5baeb8e-f776-4f51-af61-3a92680fc7ed None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:21.776914 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e7903-3624-4066-a78f-3787aaf9d339 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:47:21.781694 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-722e7903-3624-4066-a78f-3787aaf9d339 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:21.781694 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:21.781694 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:21.781694 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:21.781694 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:21.781694 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:21.781694 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:21.781694 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:21.781694 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:21.781694 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:21.781694 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:21.781694 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:21.781694 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:47:21.781694 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:47:21.781694 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:47:21.781694 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:47:21.782597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:47:21.782597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:47:21.782597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:47:21.782597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:47:21.782597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:21.782597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:21.782597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:21.782597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:21.782597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:21.782597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:21.782597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:21.782597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:21.782597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:21.782597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:21.782597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:21.782597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:21.783510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:21.783510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:21.783510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:21.783510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:21.783510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:47:21.783510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:47:21.783510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:47:21.783510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:47:21.783510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:47:21.783510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:47:21.783510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:21.783510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:21.783510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 286/580] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Sep 9 20:47:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:21.783510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e7903-3624-4066-a78f-3787aaf9d339 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:21.790672 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 295/581] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:21.791698 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5baeb8e-f776-4f51-af61-3a92680fc7ed None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:21.842625 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:21.896451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e7903-3624-4066-a78f-3787aaf9d339 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:21.900858 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5baeb8e-f776-4f51-af61-3a92680fc7ed None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:21.933371 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:21.936601 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d153a24-022c-437d-9f3b-c6d52f28d295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:21.937237 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d153a24-022c-437d-9f3b-c6d52f28d295 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:21.937522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d153a24-022c-437d-9f3b-c6d52f28d295 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:21.937791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d153a24-022c-437d-9f3b-c6d52f28d295 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:21.939794 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa5335-c7b0-49d2-b2d1-955fc10eb786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:21.940427 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa5335-c7b0-49d2-b2d1-955fc10eb786 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:21.940829 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa5335-c7b0-49d2-b2d1-955fc10eb786 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:21.941007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa5335-c7b0-49d2-b2d1-955fc10eb786 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:21.941689 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aa5335-c7b0-49d2-b2d1-955fc10eb786 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:21.941997 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aa5335-c7b0-49d2-b2d1-955fc10eb786 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:21.943418 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aa5335-c7b0-49d2-b2d1-955fc10eb786 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:21.945591 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d153a24-022c-437d-9f3b-c6d52f28d295 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:21.946936 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d153a24-022c-437d-9f3b-c6d52f28d295 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:21.958432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-9d153a24-022c-437d-9f3b-c6d52f28d295 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:21.958432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:21.958432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:21.958432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:21.958432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:21.958432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:21.958432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:21.958432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:21.958432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:21.958432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:21.958432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:21.958432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:21.958432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:47:21.958432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:47:21.958432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:47:21.959842 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:47:21.959842 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:21.959842 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:21.959842 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:21.959842 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:21.959842 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:21.959842 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:21.959842 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:21.959842 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:21.959842 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:21.959842 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:21.959842 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:21.959842 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:21.959842 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:21.959842 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:21.959842 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:21.959842 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:21.960851 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:47:21.960851 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:47:21.960851 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:47:21.960851 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:47:21.960851 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:47:21.960851 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:47:21.960851 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:21.960851 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:21.960851 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 287/582] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:47:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:21.960851 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d153a24-022c-437d-9f3b-c6d52f28d295 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:21.967022 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 296/583] 10.222.0.48 () {56 vars in 1133 bytes} [Fri Sep 9 20:47:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:21.968186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa5335-c7b0-49d2-b2d1-955fc10eb786 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:22.067134 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d153a24-022c-437d-9f3b-c6d52f28d295 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:22.076451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:22.083453 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa5335-c7b0-49d2-b2d1-955fc10eb786 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:22.100548 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e0cb0-ea53-4398-8342-c5b9867751c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:22.101709 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e0cb0-ea53-4398-8342-c5b9867751c6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:22.102057 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e0cb0-ea53-4398-8342-c5b9867751c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:22.102396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e0cb0-ea53-4398-8342-c5b9867751c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:22.103126 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e0cb0-ea53-4398-8342-c5b9867751c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:22.103501 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e0cb0-ea53-4398-8342-c5b9867751c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:22.104972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e0cb0-ea53-4398-8342-c5b9867751c6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:22.113495 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 288/584] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Sep 9 20:47:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:22.114359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e0cb0-ea53-4398-8342-c5b9867751c6 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:22.129917 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a4652-8e88-4e08-87a7-0a8b6bd83150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:22.130984 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a4652-8e88-4e08-87a7-0a8b6bd83150 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:22.131128 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a4652-8e88-4e08-87a7-0a8b6bd83150 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:22.131382 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a4652-8e88-4e08-87a7-0a8b6bd83150 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:22.138687 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a4652-8e88-4e08-87a7-0a8b6bd83150 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:22.140160 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a4652-8e88-4e08-87a7-0a8b6bd83150 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:22.152393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-1a3a4652-8e88-4e08-87a7-0a8b6bd83150 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 20:47:22.152393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:22.152393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:22.152393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:22.152393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:22.152393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:22.152393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:22.152393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:22.152393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:22.152393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:22.152393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:22.152393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:22.152393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:47:22.152393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:47:22.152393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:47:22.152393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:47:22.153524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:22.153524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:22.153524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:47:22.153524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:22.153524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:47:22.153524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:22.153524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:22.153524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:22.153524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:22.153524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:22.153524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:22.153524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:22.153524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:22.153524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:22.153524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:22.153524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:22.153524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:22.153524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:22.154656 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:22.154656 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:22.154656 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:47:22.154656 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:47:22.154656 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:47:22.154656 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:47:22.154656 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:47:22.154656 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:47:22.154656 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 20:47:22.154656 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:22.155959 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 297/585] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:47:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:22.161001 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a4652-8e88-4e08-87a7-0a8b6bd83150 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:22.215149 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:22.221223 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e0cb0-ea53-4398-8342-c5b9867751c6 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:22.255534 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a91805-8c52-4d5a-9d54-941c7e0a5a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:22.256265 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a91805-8c52-4d5a-9d54-941c7e0a5a45 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:22.256618 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a91805-8c52-4d5a-9d54-941c7e0a5a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:22.256987 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a91805-8c52-4d5a-9d54-941c7e0a5a45 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:22.257757 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a91805-8c52-4d5a-9d54-941c7e0a5a45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:22.258144 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a91805-8c52-4d5a-9d54-941c7e0a5a45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:22.259756 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a91805-8c52-4d5a-9d54-941c7e0a5a45 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:22.270341 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a4652-8e88-4e08-87a7-0a8b6bd83150 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:22.270787 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 289/586] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:47:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:22.271573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a91805-8c52-4d5a-9d54-941c7e0a5a45 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:22.306915 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:22.314041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b180de-852c-499b-b343-e2e3597295f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:22.314913 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b180de-852c-499b-b343-e2e3597295f4 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:22.319190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b180de-852c-499b-b343-e2e3597295f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:22.319190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b180de-852c-499b-b343-e2e3597295f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:22.327172 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b180de-852c-499b-b343-e2e3597295f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:22.328817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b180de-852c-499b-b343-e2e3597295f4 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:22.344012 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-f8b180de-852c-499b-b343-e2e3597295f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:22.344012 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:22.344012 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:22.344012 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:22.344012 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:22.344012 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:22.344012 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:22.344012 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:22.344012 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:22.344012 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:22.344012 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:22.344012 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:22.344012 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:22.344012 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:22.344012 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:22.344012 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:22.345135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:22.345135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:22.345135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:22.345135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:22.345135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:22.345135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:22.345135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:22.345135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:22.345135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:22.345135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:22.345135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:22.345135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:22.345135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:22.345135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:22.345135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:22.345135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:22.345135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:22.346208 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:22.346208 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:22.346208 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:22.347725 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 298/587] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:22.348668 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b180de-852c-499b-b343-e2e3597295f4 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:22.382160 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a91805-8c52-4d5a-9d54-941c7e0a5a45 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:22.417272 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e3ef2b-8b9d-4839-995c-fac33f1175ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:22.418220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3ef2b-8b9d-4839-995c-fac33f1175ac None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:22.418503 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3ef2b-8b9d-4839-995c-fac33f1175ac None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:22.418774 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3ef2b-8b9d-4839-995c-fac33f1175ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:22.429833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e3ef2b-8b9d-4839-995c-fac33f1175ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:22.431307 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e3ef2b-8b9d-4839-995c-fac33f1175ac None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:22.445500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-53e3ef2b-8b9d-4839-995c-fac33f1175ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:22.445500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:22.445500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:22.445500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:22.445500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:22.445500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:22.445500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:22.445500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:22.445500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:22.445500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:22.445500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:22.445500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:22.445500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:22.445500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:22.445500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:22.445500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:22.448143 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:22.448143 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:22.448143 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:22.448143 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:22.448143 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:22.448143 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:22.448143 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:22.448143 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:22.448143 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:22.448143 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:22.448143 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:22.448143 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:22.448143 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:22.448143 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:22.448143 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:22.448143 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:22.448143 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:22.449140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:22.449140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:22.449140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:22.449140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 290/588] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:22.449140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e3ef2b-8b9d-4839-995c-fac33f1175ac None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:22.458118 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:22.462636 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b180de-852c-499b-b343-e2e3597295f4 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:22.496968 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d724a29-6b72-40f3-a9f7-02b04b8fbd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:22.497570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d724a29-6b72-40f3-a9f7-02b04b8fbd13 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:22.497855 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d724a29-6b72-40f3-a9f7-02b04b8fbd13 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:22.498129 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d724a29-6b72-40f3-a9f7-02b04b8fbd13 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:22.498848 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d724a29-6b72-40f3-a9f7-02b04b8fbd13 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:22.499196 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d724a29-6b72-40f3-a9f7-02b04b8fbd13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:22.500601 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d724a29-6b72-40f3-a9f7-02b04b8fbd13 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:22.522991 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 299/589] 10.222.0.48 () {56 vars in 1137 bytes} [Fri Sep 9 20:47:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:22.522991 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d724a29-6b72-40f3-a9f7-02b04b8fbd13 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:22.539645 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:22.558330 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e3ef2b-8b9d-4839-995c-fac33f1175ac None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:22.595267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575b761-8437-44f0-a0ef-5306c9ab526c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:22.596242 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575b761-8437-44f0-a0ef-5306c9ab526c None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:22.596525 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575b761-8437-44f0-a0ef-5306c9ab526c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:22.596795 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575b761-8437-44f0-a0ef-5306c9ab526c None admin] PATH_INFO: `/v3/projects/91319aa782334fb9930770029b74e84a/users/7832e97d7e97439e821268d05816ae3d/roles/1e000597af164d54a40df2a39c729820` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:22.611718 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575b761-8437-44f0-a0ef-5306c9ab526c None admin] RBAC: Authorizing `identity:create_grant(project_id=91319aa782334fb9930770029b74e84a, user_id=7832e97d7e97439e821268d05816ae3d, role_id=1e000597af164d54a40df2a39c729820)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:22.613240 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575b761-8437-44f0-a0ef-5306c9ab526c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:22.628238 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d724a29-6b72-40f3-a9f7-02b04b8fbd13 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:22.665354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e7497-2936-417a-9981-5ba42e59b53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:22.665971 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e7497-2936-417a-9981-5ba42e59b53d None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:22.666296 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e7497-2936-417a-9981-5ba42e59b53d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:22.666567 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e7497-2936-417a-9981-5ba42e59b53d None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:22.667262 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e7497-2936-417a-9981-5ba42e59b53d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:22.667549 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e7497-2936-417a-9981-5ba42e59b53d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:22.668952 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e7497-2936-417a-9981-5ba42e59b53d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:22.680456 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 300/590] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:22.681239 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e7497-2936-417a-9981-5ba42e59b53d None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:22.702148 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:22.703026 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 291/591] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Sep 9 20:47:22 2022] PUT /identity/v3/projects/91319aa782334fb9930770029b74e84a/users/7832e97d7e97439e821268d05816ae3d/roles/1e000597af164d54a40df2a39c729820 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:47:22.704129 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575b761-8437-44f0-a0ef-5306c9ab526c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:22.790928 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e7497-2936-417a-9981-5ba42e59b53d None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:22.828826 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfc2ac-1b76-4dda-a960-5b2c686f8241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:22.829445 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfc2ac-1b76-4dda-a960-5b2c686f8241 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:22.829722 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfc2ac-1b76-4dda-a960-5b2c686f8241 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:22.829991 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfc2ac-1b76-4dda-a960-5b2c686f8241 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:22.830657 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfc2ac-1b76-4dda-a960-5b2c686f8241 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:22.830949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfc2ac-1b76-4dda-a960-5b2c686f8241 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:22.832371 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfc2ac-1b76-4dda-a960-5b2c686f8241 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:22.856221 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 301/592] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:22.857223 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfc2ac-1b76-4dda-a960-5b2c686f8241 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:22.861591 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575b761-8437-44f0-a0ef-5306c9ab526c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:22.894836 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:22.903093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d12719-4ee1-481a-86ce-e254b976a240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:22.904082 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d12719-4ee1-481a-86ce-e254b976a240 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:22.904082 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d12719-4ee1-481a-86ce-e254b976a240 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:22.904273 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d12719-4ee1-481a-86ce-e254b976a240 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:22.915204 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d12719-4ee1-481a-86ce-e254b976a240 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:22.917083 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d12719-4ee1-481a-86ce-e254b976a240 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:22.933231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-f6d12719-4ee1-481a-86ce-e254b976a240 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:22.933231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:22.933231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:22.933231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:22.933231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:22.933231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:22.933231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:22.933231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:22.933231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:22.933231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:22.933231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:22.933231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:22.933231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:22.933231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:22.933231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:22.933231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:22.934230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:22.934230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:22.934230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:22.934230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:22.934230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:22.934230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:22.934230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:22.934230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:22.934230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:22.934230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:22.934230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:22.934230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:22.934230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:22.934230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:22.934230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:22.934230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:22.934230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:22.935787 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:22.935787 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:22.935787 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:22.935960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 292/593] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:22.937015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d12719-4ee1-481a-86ce-e254b976a240 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:22.958811 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:22.994650 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfc2ac-1b76-4dda-a960-5b2c686f8241 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:23.037063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf3cbb-852d-47c7-b8d6-3859509a8e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:23.037793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf3cbb-852d-47c7-b8d6-3859509a8e98 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:23.038612 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf3cbb-852d-47c7-b8d6-3859509a8e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:23.038888 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf3cbb-852d-47c7-b8d6-3859509a8e98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:23.046534 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf3cbb-852d-47c7-b8d6-3859509a8e98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:23.048088 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf3cbb-852d-47c7-b8d6-3859509a8e98 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:23.061167 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-bdbf3cbb-852d-47c7-b8d6-3859509a8e98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:23.061167 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:23.061167 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:23.061167 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:23.061167 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:23.061167 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:23.061167 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:23.061167 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:23.061167 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:23.061167 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:23.061167 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:23.061167 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:23.061167 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:47:23.061167 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:47:23.061167 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:47:23.062141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:47:23.062141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:23.062141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:23.062141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:23.062141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:23.062141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:23.062141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:23.062141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:23.062141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:23.062141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:23.062141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:23.062141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:23.062141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:23.062141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:23.062141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:23.062141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:23.062141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:23.063016 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:47:23.063016 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:47:23.063016 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:47:23.063016 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:47:23.063016 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:47:23.063016 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:47:23.063016 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:23.063016 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:23.063016 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 302/594] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:47:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:23.065526 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d12719-4ee1-481a-86ce-e254b976a240 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:23.068047 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf3cbb-852d-47c7-b8d6-3859509a8e98 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:23.107421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96961972-dc64-4c2a-8635-c5a359bf7600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:23.108062 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96961972-dc64-4c2a-8635-c5a359bf7600 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:23.108360 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96961972-dc64-4c2a-8635-c5a359bf7600 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:23.108631 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96961972-dc64-4c2a-8635-c5a359bf7600 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:23.118572 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96961972-dc64-4c2a-8635-c5a359bf7600 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:23.120725 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96961972-dc64-4c2a-8635-c5a359bf7600 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:23.135449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-96961972-dc64-4c2a-8635-c5a359bf7600 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 20:47:23.135449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:23.135449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:23.135449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:23.135449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:23.135449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:23.135449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:23.135449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:23.135449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:23.135449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:23.135449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:23.135449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:23.135449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:47:23.135449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:47:23.135449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:47:23.135449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:47:23.137354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:23.137354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:23.137354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:47:23.137354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:23.137354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:47:23.137354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:23.137354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:23.137354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:23.137354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:23.137354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:23.137354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:23.137354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:23.137354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:23.137354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:23.137354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:23.137354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:23.137354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:23.137354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:23.138979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:23.138979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:23.138979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:47:23.138979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:47:23.138979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:47:23.138979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:47:23.138979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:47:23.138979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:47:23.138979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 20:47:23.138979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:23.138979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 293/595] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Sep 9 20:47:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:23.138979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96961972-dc64-4c2a-8635-c5a359bf7600 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:23.182653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf3cbb-852d-47c7-b8d6-3859509a8e98 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:23.193894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:23.227937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b9ec98-bb86-41ea-9e6d-c0289903a67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:23.228691 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9ec98-bb86-41ea-9e6d-c0289903a67e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:23.228975 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9ec98-bb86-41ea-9e6d-c0289903a67e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:23.229249 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9ec98-bb86-41ea-9e6d-c0289903a67e None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:23.229900 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b9ec98-bb86-41ea-9e6d-c0289903a67e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:23.230564 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b9ec98-bb86-41ea-9e6d-c0289903a67e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:23.232588 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b9ec98-bb86-41ea-9e6d-c0289903a67e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:23.241949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:23.252157 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 303/596] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:23.253294 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b9ec98-bb86-41ea-9e6d-c0289903a67e None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:23.263872 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96961972-dc64-4c2a-8635-c5a359bf7600 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:23.301953 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b6a34-aebd-4c87-b90e-b9abba929714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:23.302946 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b6a34-aebd-4c87-b90e-b9abba929714 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:23.303384 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b6a34-aebd-4c87-b90e-b9abba929714 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:23.303834 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b6a34-aebd-4c87-b90e-b9abba929714 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:23.304900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7b6a34-aebd-4c87-b90e-b9abba929714 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:23.305355 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7b6a34-aebd-4c87-b90e-b9abba929714 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:23.307075 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7b6a34-aebd-4c87-b90e-b9abba929714 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:23.320707 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 294/597] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:47:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:23.325158 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b6a34-aebd-4c87-b90e-b9abba929714 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:23.376519 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b9ec98-bb86-41ea-9e6d-c0289903a67e None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:23.421609 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b981d-11f1-4161-8c77-956235980696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:23.422217 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b981d-11f1-4161-8c77-956235980696 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:23.422493 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b981d-11f1-4161-8c77-956235980696 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:23.422762 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b981d-11f1-4161-8c77-956235980696 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:23.423486 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37b981d-11f1-4161-8c77-956235980696 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:23.423789 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37b981d-11f1-4161-8c77-956235980696 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:23.425802 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37b981d-11f1-4161-8c77-956235980696 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:23.438276 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b6a34-aebd-4c87-b90e-b9abba929714 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:23.441724 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:23.453819 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 304/598] 10.222.0.48 () {56 vars in 1139 bytes} [Fri Sep 9 20:47:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:23.453819 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b981d-11f1-4161-8c77-956235980696 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:23.482947 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85782598-b9bc-4cc2-9f0f-3bab18d8ef58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:23.484088 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85782598-b9bc-4cc2-9f0f-3bab18d8ef58 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:23.484751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85782598-b9bc-4cc2-9f0f-3bab18d8ef58 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:23.485242 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85782598-b9bc-4cc2-9f0f-3bab18d8ef58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:23.493571 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85782598-b9bc-4cc2-9f0f-3bab18d8ef58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:23.495579 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85782598-b9bc-4cc2-9f0f-3bab18d8ef58 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:23.510817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-85782598-b9bc-4cc2-9f0f-3bab18d8ef58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:23.510817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:23.510817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:23.510817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:23.510817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:23.510817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:23.510817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:23.510817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:23.510817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:23.510817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:23.510817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:23.510817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:23.510817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:47:23.510817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:47:23.510817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:47:23.511827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:47:23.511827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:23.511827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:23.511827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:23.511827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:23.511827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:23.511827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:23.511827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:23.511827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:23.511827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:23.511827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:23.511827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:23.511827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:23.511827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:23.511827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:23.511827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:23.511827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:23.512750 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:47:23.512750 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:47:23.512750 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:47:23.512750 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:47:23.512750 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:47:23.512750 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:47:23.512750 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:23.512750 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:23.521433 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:23.522399 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 295/599] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:47:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:23.524279 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85782598-b9bc-4cc2-9f0f-3bab18d8ef58 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:23.582690 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b981d-11f1-4161-8c77-956235980696 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:23.631076 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118feea-8e2a-4a7c-928a-b02a3dde673d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:23.632627 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118feea-8e2a-4a7c-928a-b02a3dde673d None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:23.633052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118feea-8e2a-4a7c-928a-b02a3dde673d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:23.633468 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118feea-8e2a-4a7c-928a-b02a3dde673d None admin] PATH_INFO: `/v3/projects/91319aa782334fb9930770029b74e84a/users/613b845a9b4d432daf8d478fc3970708/roles/1e000597af164d54a40df2a39c729820` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:23.650916 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85782598-b9bc-4cc2-9f0f-3bab18d8ef58 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:23.656672 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2118feea-8e2a-4a7c-928a-b02a3dde673d None admin] RBAC: Authorizing `identity:create_grant(project_id=91319aa782334fb9930770029b74e84a, user_id=613b845a9b4d432daf8d478fc3970708, role_id=1e000597af164d54a40df2a39c729820)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:23.658117 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2118feea-8e2a-4a7c-928a-b02a3dde673d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:23.689912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca429651-ddff-429a-a525-b040f62f75a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:23.690461 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca429651-ddff-429a-a525-b040f62f75a8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:23.690743 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca429651-ddff-429a-a525-b040f62f75a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:23.691016 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca429651-ddff-429a-a525-b040f62f75a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:23.702894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca429651-ddff-429a-a525-b040f62f75a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:23.704429 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca429651-ddff-429a-a525-b040f62f75a8 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:23.720332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-ca429651-ddff-429a-a525-b040f62f75a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:23.720332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:23.720332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:23.720332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:23.720332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:23.720332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:23.720332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:23.720332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:23.720332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:23.720332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:23.720332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:23.720332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:23.720332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:23.720332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:23.720332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:23.720332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:23.725925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:23.725925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:23.725925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:23.725925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:23.725925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:23.725925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:23.725925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:23.725925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:23.725925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:23.725925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:23.725925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:23.725925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:23.725925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:23.725925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:23.725925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:23.725925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:23.725925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:23.729003 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:23.729003 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:23.729003 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:23.729003 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 296/600] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:23.729003 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca429651-ddff-429a-a525-b040f62f75a8 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:23.730960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 305/601] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Sep 9 20:47:23 2022] PUT /identity/v3/projects/91319aa782334fb9930770029b74e84a/users/613b845a9b4d432daf8d478fc3970708/roles/1e000597af164d54a40df2a39c729820 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:47:23.740227 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2b7ca-0a5c-4b3b-b523-160786f4be61 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:23.740227 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2b7ca-0a5c-4b3b-b523-160786f4be61 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:23.740227 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2b7ca-0a5c-4b3b-b523-160786f4be61 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:23.740227 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 306/602] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:23.740227 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb2b7ca-0a5c-4b3b-b523-160786f4be61 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:23.755813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:23.864654 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb2b7ca-0a5c-4b3b-b523-160786f4be61 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:23.874620 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:23.876309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca429651-ddff-429a-a525-b040f62f75a8 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:23.910860 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de58a48-4651-4ce0-a174-b1916a5d4496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:23.912308 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de58a48-4651-4ce0-a174-b1916a5d4496 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:23.912839 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de58a48-4651-4ce0-a174-b1916a5d4496 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:23.913285 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de58a48-4651-4ce0-a174-b1916a5d4496 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:23.914097 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de58a48-4651-4ce0-a174-b1916a5d4496 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:23.914541 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de58a48-4651-4ce0-a174-b1916a5d4496 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:23.918271 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de58a48-4651-4ce0-a174-b1916a5d4496 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:23.925020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffe8a6-2f80-48f8-8a8f-eddd019da4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:23.925852 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffe8a6-2f80-48f8-8a8f-eddd019da4ed None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:23.926135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffe8a6-2f80-48f8-8a8f-eddd019da4ed None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:23.926414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffe8a6-2f80-48f8-8a8f-eddd019da4ed None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:23.927127 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffe8a6-2f80-48f8-8a8f-eddd019da4ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:23.927436 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffe8a6-2f80-48f8-8a8f-eddd019da4ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:23.929452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffe8a6-2f80-48f8-8a8f-eddd019da4ed None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:23.930397 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 307/603] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:23.931780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e310a5-2fc7-48b2-9777-511677d8598d None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:23.932427 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e310a5-2fc7-48b2-9777-511677d8598d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:23.932845 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e310a5-2fc7-48b2-9777-511677d8598d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:23.951910 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 297/604] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:47:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:23.952810 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffe8a6-2f80-48f8-8a8f-eddd019da4ed None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:23.954924 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-84e310a5-2fc7-48b2-9777-511677d8598d None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:23.956757 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-84e310a5-2fc7-48b2-9777-511677d8598d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:24.032955 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 308/605] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:47:24.033872 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e310a5-2fc7-48b2-9777-511677d8598d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:24.040433 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:24.064773 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffe8a6-2f80-48f8-8a8f-eddd019da4ed None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:24.131686 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc347ddd-7e55-4bd4-83c4-6d82d41d91c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:24.133160 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc347ddd-7e55-4bd4-83c4-6d82d41d91c8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:24.133597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc347ddd-7e55-4bd4-83c4-6d82d41d91c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:24.134051 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc347ddd-7e55-4bd4-83c4-6d82d41d91c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:24.137086 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:24.143291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e310a5-2fc7-48b2-9777-511677d8598d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:24.149182 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc347ddd-7e55-4bd4-83c4-6d82d41d91c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:24.150684 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc347ddd-7e55-4bd4-83c4-6d82d41d91c8 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:24.173996 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-bc347ddd-7e55-4bd4-83c4-6d82d41d91c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:24.173996 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:24.173996 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:24.173996 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:24.173996 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:24.173996 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:24.173996 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:24.173996 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:24.173996 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:24.173996 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:24.173996 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:24.173996 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:24.173996 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:47:24.173996 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:47:24.173996 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:47:24.175082 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:47:24.175082 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:24.175082 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:24.175082 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:24.175082 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:24.175082 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:24.175082 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:24.175082 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:24.175082 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:24.175082 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:24.175082 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:24.175082 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:24.175082 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:24.175082 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:24.175082 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:24.175082 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:24.175082 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:24.176043 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:47:24.176043 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:47:24.176043 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:47:24.176043 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:47:24.176043 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:47:24.176043 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:47:24.176043 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:24.176043 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:24.180159 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 298/606] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:47:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:24.182518 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc347ddd-7e55-4bd4-83c4-6d82d41d91c8 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:24.186669 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0788a-9e7a-4350-ada8-32d36c79584c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:24.187918 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0788a-9e7a-4350-ada8-32d36c79584c None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:24.188523 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0788a-9e7a-4350-ada8-32d36c79584c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:24.189044 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0788a-9e7a-4350-ada8-32d36c79584c None admin] PATH_INFO: `/v3/projects/91319aa782334fb9930770029b74e84a/users/cf91131ee5bb481bbb47097cad0c003a/roles/1e000597af164d54a40df2a39c729820` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:24.206557 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd0788a-9e7a-4350-ada8-32d36c79584c None admin] RBAC: Authorizing `identity:create_grant(project_id=91319aa782334fb9930770029b74e84a, user_id=cf91131ee5bb481bbb47097cad0c003a, role_id=1e000597af164d54a40df2a39c729820)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:24.208407 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd0788a-9e7a-4350-ada8-32d36c79584c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:24.302421 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 309/607] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Sep 9 20:47:24 2022] PUT /identity/v3/projects/91319aa782334fb9930770029b74e84a/users/cf91131ee5bb481bbb47097cad0c003a/roles/1e000597af164d54a40df2a39c729820 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:47:24.321218 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc347ddd-7e55-4bd4-83c4-6d82d41d91c8 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:24.362540 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128011bc-31d6-417c-8fd4-694873f4719b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:24.363560 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128011bc-31d6-417c-8fd4-694873f4719b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:24.364055 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128011bc-31d6-417c-8fd4-694873f4719b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:24.364620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128011bc-31d6-417c-8fd4-694873f4719b None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:24.365648 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128011bc-31d6-417c-8fd4-694873f4719b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:24.366790 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128011bc-31d6-417c-8fd4-694873f4719b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:24.368635 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128011bc-31d6-417c-8fd4-694873f4719b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:24.369114 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0788a-9e7a-4350-ada8-32d36c79584c None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:24.372262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:24.384401 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 299/608] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:47:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:47:24.391079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128011bc-31d6-417c-8fd4-694873f4719b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:24.544837 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0788a-9e7a-4350-ada8-32d36c79584c None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:24.546091 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128011bc-31d6-417c-8fd4-694873f4719b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:24.582834 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292916e1-de3c-415a-86be-ecbd8e47d8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:24.583563 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292916e1-de3c-415a-86be-ecbd8e47d8f3 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:24.583842 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292916e1-de3c-415a-86be-ecbd8e47d8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:24.584601 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292916e1-de3c-415a-86be-ecbd8e47d8f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:24.590602 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855fb684-a254-47c7-83f1-48f1d61b284a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:24.591414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fb684-a254-47c7-83f1-48f1d61b284a None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:24.591692 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fb684-a254-47c7-83f1-48f1d61b284a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:24.592497 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fb684-a254-47c7-83f1-48f1d61b284a None admin] PATH_INFO: `/v3/projects/91319aa782334fb9930770029b74e84a/users/8f5dfac06ac9492a86503ba133c638a9/roles/1e000597af164d54a40df2a39c729820` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:24.597112 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292916e1-de3c-415a-86be-ecbd8e47d8f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:47:24.602822 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-292916e1-de3c-415a-86be-ecbd8e47d8f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:24.602822 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:24.602822 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:24.602822 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:24.602822 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:24.602822 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:24.602822 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:24.602822 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:24.602822 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:24.602822 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:24.602822 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:24.602822 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:24.602822 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:47:24.602822 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:47:24.602822 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:47:24.602822 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:47:24.603763 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:47:24.603763 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:47:24.603763 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:47:24.603763 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:47:24.603763 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:24.603763 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:24.603763 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:24.603763 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:24.603763 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:24.603763 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:24.603763 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:24.603763 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:24.603763 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:24.603763 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:24.603763 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:24.603763 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:24.604662 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:24.604662 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:24.604662 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:24.604662 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:24.604662 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:47:24.604662 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:47:24.604662 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:47:24.604662 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:47:24.604662 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:47:24.604662 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:47:24.604662 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:24.604662 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:24.604662 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 310/609] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Sep 9 20:47:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:24.608929 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855fb684-a254-47c7-83f1-48f1d61b284a None admin] RBAC: Authorizing `identity:create_grant(project_id=91319aa782334fb9930770029b74e84a, user_id=8f5dfac06ac9492a86503ba133c638a9, role_id=1e000597af164d54a40df2a39c729820)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:24.610451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855fb684-a254-47c7-83f1-48f1d61b284a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:24.613240 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292916e1-de3c-415a-86be-ecbd8e47d8f3 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:24.630002 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:24.690318 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 300/610] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Sep 9 20:47:24 2022] PUT /identity/v3/projects/91319aa782334fb9930770029b74e84a/users/8f5dfac06ac9492a86503ba133c638a9/roles/1e000597af164d54a40df2a39c729820 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:47:24.739390 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292916e1-de3c-415a-86be-ecbd8e47d8f3 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:24.776287 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6601bbe-7bb3-49ed-aef4-4521270de91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:24.776916 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6601bbe-7bb3-49ed-aef4-4521270de91d None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:24.777196 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6601bbe-7bb3-49ed-aef4-4521270de91d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:24.777471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6601bbe-7bb3-49ed-aef4-4521270de91d None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:24.778161 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6601bbe-7bb3-49ed-aef4-4521270de91d None admin] RBAC: Adding query filter params (name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:24.778467 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6601bbe-7bb3-49ed-aef4-4521270de91d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:24.779906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6601bbe-7bb3-49ed-aef4-4521270de91d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:24.790052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 311/611] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Sep 9 20:47:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:24.797285 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855fb684-a254-47c7-83f1-48f1d61b284a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:24.873350 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:24.943919 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855fb684-a254-47c7-83f1-48f1d61b284a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:24.987669 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7bf75-4b07-426e-8b5d-a183fb730bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:24.988398 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7bf75-4b07-426e-8b5d-a183fb730bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:24.988944 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7bf75-4b07-426e-8b5d-a183fb730bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:24.988944 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7bf75-4b07-426e-8b5d-a183fb730bb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:25.002405 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c7bf75-4b07-426e-8b5d-a183fb730bb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:25.003866 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c7bf75-4b07-426e-8b5d-a183fb730bb5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:25.019702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-c8c7bf75-4b07-426e-8b5d-a183fb730bb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:25.019702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:25.019702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:25.019702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:25.019702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:25.019702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:25.019702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:25.019702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:25.019702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:25.019702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:25.019702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:25.019702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:25.019702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:25.019702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:25.019702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:25.019702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:25.020686 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:25.020686 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:25.020686 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:25.020686 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:25.020686 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:25.020686 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:25.020686 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:25.020686 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:25.020686 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:25.020686 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:25.020686 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:25.020686 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:25.020686 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:25.020686 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:25.020686 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:25.020686 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:25.020686 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:25.021607 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:25.021607 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:25.021607 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:25.021607 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 301/612] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:25.027883 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6601bbe-7bb3-49ed-aef4-4521270de91d None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:25.115524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:25.151009 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6601bbe-7bb3-49ed-aef4-4521270de91d None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:25.185741 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa854f-1e95-4b3c-a4fa-2defef5ad56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:25.186570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa854f-1e95-4b3c-a4fa-2defef5ad56e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:25.186907 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa854f-1e95-4b3c-a4fa-2defef5ad56e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:25.187444 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa854f-1e95-4b3c-a4fa-2defef5ad56e None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:25.187825 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa854f-1e95-4b3c-a4fa-2defef5ad56e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:25.188238 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa854f-1e95-4b3c-a4fa-2defef5ad56e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:25.189533 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa854f-1e95-4b3c-a4fa-2defef5ad56e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:25.202457 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 312/613] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:25.207812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7bf75-4b07-426e-8b5d-a183fb730bb5 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:25.258813 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ffa54f-9024-4e31-a78f-c5b2ffe9dd93 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:25.259153 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ffa54f-9024-4e31-a78f-c5b2ffe9dd93 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:25.259359 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ffa54f-9024-4e31-a78f-c5b2ffe9dd93 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:25.262077 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 313/614] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:25.269036 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fcac95-db48-4ae3-abe5-5accbc1d95d0 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:25.270318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fcac95-db48-4ae3-abe5-5accbc1d95d0 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:25.270821 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fcac95-db48-4ae3-abe5-5accbc1d95d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:25.305972 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-c4fcac95-db48-4ae3-abe5-5accbc1d95d0 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:25.307674 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c4fcac95-db48-4ae3-abe5-5accbc1d95d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:25.368427 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7bf75-4b07-426e-8b5d-a183fb730bb5 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:25.372264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:25.407782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d5a36-9f7c-49c9-9956-31a87e6cefe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:25.410634 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d5a36-9f7c-49c9-9956-31a87e6cefe5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:25.410634 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d5a36-9f7c-49c9-9956-31a87e6cefe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:25.410634 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d5a36-9f7c-49c9-9956-31a87e6cefe5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:25.422452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d5a36-9f7c-49c9-9956-31a87e6cefe5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:25.423934 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d5a36-9f7c-49c9-9956-31a87e6cefe5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:25.448552 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-5b3d5a36-9f7c-49c9-9956-31a87e6cefe5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 20:47:25.448552 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:25.448552 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:25.448552 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:25.448552 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:25.448552 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:25.448552 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:25.448552 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:25.448552 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:25.448552 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:25.448552 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:25.448552 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:25.448552 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:47:25.448552 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:47:25.448552 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:47:25.448552 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:47:25.449544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:25.449544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:25.449544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:47:25.449544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:25.449544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:47:25.449544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:25.449544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:25.449544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:25.449544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:25.449544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:25.449544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:25.449544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:25.449544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:25.449544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:25.449544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:25.449544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:25.449544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:25.449544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:25.450488 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:25.450488 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:25.450488 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:47:25.450488 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:47:25.450488 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:47:25.450488 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:47:25.450488 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:47:25.450488 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:47:25.450488 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 20:47:25.450488 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:25.450488 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 302/615] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:47:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:25.461537 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d5a36-9f7c-49c9-9956-31a87e6cefe5 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:25.479310 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 314/616] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:47:25.590221 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d5a36-9f7c-49c9-9956-31a87e6cefe5 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:25.635682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b5ab0-d0f8-46e7-a110-de81e81760f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:25.636264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b5ab0-d0f8-46e7-a110-de81e81760f1 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:25.636553 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b5ab0-d0f8-46e7-a110-de81e81760f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:25.636829 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b5ab0-d0f8-46e7-a110-de81e81760f1 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:25.637871 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6b5ab0-d0f8-46e7-a110-de81e81760f1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:25.638188 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6b5ab0-d0f8-46e7-a110-de81e81760f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:25.639796 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6b5ab0-d0f8-46e7-a110-de81e81760f1 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:25.660137 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 303/617] 10.222.0.48 () {56 vars in 1135 bytes} [Fri Sep 9 20:47:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:25.665934 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fcac95-db48-4ae3-abe5-5accbc1d95d0 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:25.728826 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:25.779241 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fcac95-db48-4ae3-abe5-5accbc1d95d0 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:25.814489 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb115fc-4e8e-453f-b640-a7bb62f5700f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:25.815126 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb115fc-4e8e-453f-b640-a7bb62f5700f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:25.815405 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb115fc-4e8e-453f-b640-a7bb62f5700f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:25.815677 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb115fc-4e8e-453f-b640-a7bb62f5700f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:25.826545 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb115fc-4e8e-453f-b640-a7bb62f5700f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:25.827990 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb115fc-4e8e-453f-b640-a7bb62f5700f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:25.843045 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-ceb115fc-4e8e-453f-b640-a7bb62f5700f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:25.843045 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:25.843045 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:25.843045 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:25.843045 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:25.843045 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:25.843045 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:25.843045 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:25.843045 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:25.843045 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:25.843045 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:25.843045 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:25.843045 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:25.843045 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:25.843045 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:25.843045 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:25.844005 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:25.844005 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:25.844005 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:25.844005 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:25.844005 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:25.844005 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:25.844005 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:25.844005 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:25.844005 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:25.844005 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:25.844005 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:25.844005 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:25.844005 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:25.844005 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:25.844005 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:25.844005 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:25.844005 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:25.845068 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:25.845068 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:25.845068 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:25.845068 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 315/618] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:25.850185 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b5ab0-d0f8-46e7-a110-de81e81760f1 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:25.860366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb115fc-4e8e-453f-b640-a7bb62f5700f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:25.880417 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:26.045017 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b5ab0-d0f8-46e7-a110-de81e81760f1 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:26.058473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb115fc-4e8e-453f-b640-a7bb62f5700f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:26.076124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:26.114435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b0e85-211c-4161-9bac-d15d02217472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:26.115052 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b0e85-211c-4161-9bac-d15d02217472 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:26.116845 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b0e85-211c-4161-9bac-d15d02217472 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:26.117130 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b0e85-211c-4161-9bac-d15d02217472 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:26.117784 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b0e85-211c-4161-9bac-d15d02217472 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:26.118091 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b0e85-211c-4161-9bac-d15d02217472 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:26.119469 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b0e85-211c-4161-9bac-d15d02217472 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:26.122615 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02017e1-2dca-40ef-ba9a-c15d5361ab51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:26.123253 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02017e1-2dca-40ef-ba9a-c15d5361ab51 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:26.123532 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02017e1-2dca-40ef-ba9a-c15d5361ab51 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:26.123803 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02017e1-2dca-40ef-ba9a-c15d5361ab51 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:26.138129 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 304/619] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:26.143725 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b0e85-211c-4161-9bac-d15d02217472 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:26.165337 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02017e1-2dca-40ef-ba9a-c15d5361ab51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:47:26.176611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-f02017e1-2dca-40ef-ba9a-c15d5361ab51 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:26.176611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:26.176611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:26.176611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:26.176611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:26.176611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:26.176611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:26.176611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:26.176611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:26.176611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:26.176611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:26.176611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:26.176611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:47:26.176611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:47:26.176611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:47:26.176611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:47:26.177599 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:47:26.177599 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:47:26.177599 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:47:26.177599 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:47:26.177599 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:26.177599 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:26.177599 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:26.177599 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:26.177599 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:26.177599 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:26.177599 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:26.177599 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:26.177599 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:26.177599 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:26.177599 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:26.177599 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:26.178428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:26.178428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:26.178428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:26.178428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:26.178428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:47:26.178428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:47:26.178428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:47:26.178428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:47:26.178428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:47:26.178428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:47:26.178428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:26.178428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:26.179705 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 316/620] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Sep 9 20:47:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:26.186771 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02017e1-2dca-40ef-ba9a-c15d5361ab51 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:26.304686 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:26.347964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b0e85-211c-4161-9bac-d15d02217472 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:26.379047 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02017e1-2dca-40ef-ba9a-c15d5361ab51 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:26.386501 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19464ac1-71df-479b-bdde-f69dca1ae801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:26.387734 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19464ac1-71df-479b-bdde-f69dca1ae801 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:26.388280 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19464ac1-71df-479b-bdde-f69dca1ae801 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:26.388760 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19464ac1-71df-479b-bdde-f69dca1ae801 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:26.397181 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19464ac1-71df-479b-bdde-f69dca1ae801 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:26.398991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19464ac1-71df-479b-bdde-f69dca1ae801 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:26.411515 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-19464ac1-71df-479b-bdde-f69dca1ae801 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:26.411515 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:26.411515 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:26.411515 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:26.411515 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:26.411515 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:26.411515 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:26.411515 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:26.411515 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:26.411515 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:26.411515 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:26.411515 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:26.411515 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:47:26.411515 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:47:26.411515 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:47:26.412485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:47:26.412485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:26.412485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:26.412485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:26.412485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:26.412485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:26.412485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:26.412485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:26.412485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:26.412485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:26.412485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:26.412485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:26.412485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:26.412485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:26.412485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:26.412485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:26.412485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:26.413948 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:47:26.413948 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:47:26.413948 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:47:26.413948 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:47:26.413948 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:47:26.413948 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:47:26.413948 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:26.413948 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:26.416035 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 305/621] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:47:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:26.417378 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e3818-4155-4bae-9f34-7f039f776129 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:26.417810 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e3818-4155-4bae-9f34-7f039f776129 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:26.418231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e3818-4155-4bae-9f34-7f039f776129 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:26.418343 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f2fb8-05d0-413a-af98-09fc9d85dfd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:26.419034 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f2fb8-05d0-413a-af98-09fc9d85dfd8 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:26.419417 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f2fb8-05d0-413a-af98-09fc9d85dfd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:26.419797 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f2fb8-05d0-413a-af98-09fc9d85dfd8 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:26.420444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 306/622] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:26.428229 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64f2fb8-05d0-413a-af98-09fc9d85dfd8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:26.428229 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64f2fb8-05d0-413a-af98-09fc9d85dfd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:26.428229 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64f2fb8-05d0-413a-af98-09fc9d85dfd8 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:26.428564 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e3818-4155-4bae-9f34-7f039f776129 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:26.438990 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 317/623] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Sep 9 20:47:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:26.440287 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae6407-270f-4f0c-9873-73d04a1c01fa None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:26.440575 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae6407-270f-4f0c-9873-73d04a1c01fa None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:26.440846 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae6407-270f-4f0c-9873-73d04a1c01fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:26.460697 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-acae6407-270f-4f0c-9873-73d04a1c01fa None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:26.462498 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-acae6407-270f-4f0c-9873-73d04a1c01fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:26.490823 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:26.541272 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e3818-4155-4bae-9f34-7f039f776129 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:26.544157 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 318/624] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:47:26.545031 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae6407-270f-4f0c-9873-73d04a1c01fa None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:26.604810 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6d810-f37a-474e-9be5-23ad7cf74670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:26.605339 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6d810-f37a-474e-9be5-23ad7cf74670 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:26.605621 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6d810-f37a-474e-9be5-23ad7cf74670 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:26.605895 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6d810-f37a-474e-9be5-23ad7cf74670 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:26.606599 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef6d810-f37a-474e-9be5-23ad7cf74670 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:26.606906 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef6d810-f37a-474e-9be5-23ad7cf74670 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:26.608366 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef6d810-f37a-474e-9be5-23ad7cf74670 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:26.618816 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:26.638744 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 307/625] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:47:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:26.639617 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6d810-f37a-474e-9be5-23ad7cf74670 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:26.725349 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae6407-270f-4f0c-9873-73d04a1c01fa None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:26.798990 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebaf04-8015-4f5b-8237-d15d01f88fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:26.799636 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebaf04-8015-4f5b-8237-d15d01f88fdd None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:26.799919 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebaf04-8015-4f5b-8237-d15d01f88fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:26.800216 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebaf04-8015-4f5b-8237-d15d01f88fdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:26.820851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ebaf04-8015-4f5b-8237-d15d01f88fdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:26.822309 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ebaf04-8015-4f5b-8237-d15d01f88fdd None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:26.828570 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6d810-f37a-474e-9be5-23ad7cf74670 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:26.849339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-39ebaf04-8015-4f5b-8237-d15d01f88fdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:26.849339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:26.849339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:26.849339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:26.849339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:26.849339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:26.849339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:26.849339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:26.849339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:26.849339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:26.849339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:26.849339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:26.849339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:26.849339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:26.849339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:26.849339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:26.850347 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:26.850347 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:26.850347 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:26.850347 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:26.850347 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:26.850347 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:26.850347 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:26.850347 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:26.850347 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:26.850347 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:26.850347 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:26.850347 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:26.850347 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:26.850347 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:26.850347 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:26.850347 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:26.850347 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:26.851895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:26.851895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:26.851895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:26.851895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 319/626] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:26.859019 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebaf04-8015-4f5b-8237-d15d01f88fdd None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:26.865887 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0aa54-3763-4caa-9892-a4b826c90f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:26.866709 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0aa54-3763-4caa-9892-a4b826c90f06 None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:26.866991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0aa54-3763-4caa-9892-a4b826c90f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:26.867362 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0aa54-3763-4caa-9892-a4b826c90f06 None admin] PATH_INFO: `/v3/projects/91319aa782334fb9930770029b74e84a/users/7d28fa5d49d44120bd5eaec2d4e5fdbd/roles/1e000597af164d54a40df2a39c729820` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:26.884369 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b0aa54-3763-4caa-9892-a4b826c90f06 None admin] RBAC: Authorizing `identity:create_grant(project_id=91319aa782334fb9930770029b74e84a, user_id=7d28fa5d49d44120bd5eaec2d4e5fdbd, role_id=1e000597af164d54a40df2a39c729820)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:26.885666 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:26.886861 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b0aa54-3763-4caa-9892-a4b826c90f06 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:26.943249 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 308/627] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Sep 9 20:47:26 2022] PUT /identity/v3/projects/91319aa782334fb9930770029b74e84a/users/7d28fa5d49d44120bd5eaec2d4e5fdbd/roles/1e000597af164d54a40df2a39c729820 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:47:26.945959 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7826b2-c90b-4c0f-8f67-e2f1f720e95f None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:26.946289 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7826b2-c90b-4c0f-8f67-e2f1f720e95f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:26.946558 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7826b2-c90b-4c0f-8f67-e2f1f720e95f None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:26.948125 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 309/628] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:26.948898 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7826b2-c90b-4c0f-8f67-e2f1f720e95f None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:26.996943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebaf04-8015-4f5b-8237-d15d01f88fdd None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:27.025671 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:27.042421 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d841143-46cf-4b64-929d-c118a6e6421a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:27.043339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d841143-46cf-4b64-929d-c118a6e6421a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:27.043767 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d841143-46cf-4b64-929d-c118a6e6421a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:27.044227 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d841143-46cf-4b64-929d-c118a6e6421a None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:27.045041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d841143-46cf-4b64-929d-c118a6e6421a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:27.045501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d841143-46cf-4b64-929d-c118a6e6421a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:27.047080 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d841143-46cf-4b64-929d-c118a6e6421a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:27.062774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 320/629] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:27.064420 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a23f2b-de8e-473b-b893-034256171394 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:27.064938 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a23f2b-de8e-473b-b893-034256171394 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:27.065360 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a23f2b-de8e-473b-b893-034256171394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:27.087290 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-94a23f2b-de8e-473b-b893-034256171394 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:27.088829 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-94a23f2b-de8e-473b-b893-034256171394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:27.104425 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7826b2-c90b-4c0f-8f67-e2f1f720e95f None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:27.155244 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5f98c-e1b0-487c-812d-ed3fa9403f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:27.156306 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5f98c-e1b0-487c-812d-ed3fa9403f52 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:27.156889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5f98c-e1b0-487c-812d-ed3fa9403f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:27.157368 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5f98c-e1b0-487c-812d-ed3fa9403f52 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:27.168713 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e5f98c-e1b0-487c-812d-ed3fa9403f52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:47:27.173200 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 321/630] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:47:27.174017 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-89e5f98c-e1b0-487c-812d-ed3fa9403f52 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:27.174017 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:27.174017 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:27.174017 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:27.174017 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:27.174017 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:27.174017 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:27.174017 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:27.174017 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:27.174017 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:27.174017 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:27.174017 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:27.174017 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:47:27.174017 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:47:27.174017 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:47:27.174017 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:47:27.174964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:47:27.174964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:47:27.174964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:47:27.174964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:47:27.174964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:27.174964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:27.174964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:27.174964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:27.174964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:27.174964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:27.174964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:27.174964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:27.174964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:27.174964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:27.174964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:27.174964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:27.175801 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:27.175801 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:27.175801 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:27.175801 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:27.175801 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:47:27.175801 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:47:27.175801 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:47:27.175801 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:47:27.175801 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:47:27.175801 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:47:27.175801 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:27.175801 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:27.176491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a23f2b-de8e-473b-b893-034256171394 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:27.178060 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 310/631] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Sep 9 20:47:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:27.178683 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:27.184020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5f98c-e1b0-487c-812d-ed3fa9403f52 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:27.333233 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5f98c-e1b0-487c-812d-ed3fa9403f52 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:27.344977 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:27.373493 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a23f2b-de8e-473b-b893-034256171394 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:27.397536 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7836780a-2195-4d7c-8407-caaffb473e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:27.398210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836780a-2195-4d7c-8407-caaffb473e42 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:27.398422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836780a-2195-4d7c-8407-caaffb473e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:27.398692 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836780a-2195-4d7c-8407-caaffb473e42 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:27.399362 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7836780a-2195-4d7c-8407-caaffb473e42 None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:27.399661 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7836780a-2195-4d7c-8407-caaffb473e42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:27.401067 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7836780a-2195-4d7c-8407-caaffb473e42 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:27.416563 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 311/632] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Sep 9 20:47:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:27.423149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7836780a-2195-4d7c-8407-caaffb473e42 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:27.434226 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5c9ed-6940-4edd-947a-cd4eeb32c958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:27.434877 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5c9ed-6940-4edd-947a-cd4eeb32c958 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:27.435158 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5c9ed-6940-4edd-947a-cd4eeb32c958 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:27.435690 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5c9ed-6940-4edd-947a-cd4eeb32c958 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:27.447334 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf5c9ed-6940-4edd-947a-cd4eeb32c958 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:27.451085 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf5c9ed-6940-4edd-947a-cd4eeb32c958 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:27.464124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-fbf5c9ed-6940-4edd-947a-cd4eeb32c958 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 20:47:27.464124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:27.464124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:27.464124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:27.464124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:27.464124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:27.464124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:27.464124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:27.464124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:27.464124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:27.464124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:27.464124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:27.464124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:47:27.464124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:47:27.464124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:47:27.464124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:47:27.465129 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:27.465129 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:27.465129 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:47:27.465129 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:27.465129 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:47:27.465129 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:27.465129 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:27.465129 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:27.465129 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:27.465129 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:27.465129 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:27.465129 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:27.465129 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:27.465129 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:27.465129 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:27.465129 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:27.465129 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:27.465129 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:27.466070 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:27.466070 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:27.466070 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:47:27.466070 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:47:27.466070 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:47:27.466070 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:47:27.466070 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:47:27.466070 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:47:27.466070 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 20:47:27.466070 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:27.466070 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 322/633] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:47:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:27.466663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea7574-dbbc-4c45-9938-4acb3af222b5 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:27.466879 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea7574-dbbc-4c45-9938-4acb3af222b5 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:27.468655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea7574-dbbc-4c45-9938-4acb3af222b5 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:27.468655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 323/634] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:27.473916 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea7574-dbbc-4c45-9938-4acb3af222b5 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:27.543196 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:27.554074 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7836780a-2195-4d7c-8407-caaffb473e42 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:27.586319 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea7574-dbbc-4c45-9938-4acb3af222b5 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:27.588021 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e236a-88de-45b5-a714-b5a9492d6822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:27.588684 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e236a-88de-45b5-a714-b5a9492d6822 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:27.589017 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e236a-88de-45b5-a714-b5a9492d6822 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:27.589290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e236a-88de-45b5-a714-b5a9492d6822 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:27.599646 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e236a-88de-45b5-a714-b5a9492d6822 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:27.601246 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e236a-88de-45b5-a714-b5a9492d6822 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:27.615298 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-fb7e236a-88de-45b5-a714-b5a9492d6822 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:27.615298 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:27.615298 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:27.615298 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:27.615298 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:27.615298 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:27.615298 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:27.615298 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:27.615298 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:27.615298 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:27.615298 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:27.615298 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:27.615298 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:27.615298 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:27.615298 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:27.615298 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:27.616932 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:27.616932 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:27.616932 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:27.616932 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:27.616932 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:27.616932 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:27.616932 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:27.616932 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:27.616932 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:27.616932 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:27.616932 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:27.616932 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:27.616932 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:27.616932 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:27.616932 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:27.616932 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:27.616932 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:27.617824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:27.617824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:27.617824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:27.621681 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 312/635] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:47:27.622642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a62a5-55d0-4aba-a402-01471b9863d9 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:27.622919 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a62a5-55d0-4aba-a402-01471b9863d9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:27.623248 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a62a5-55d0-4aba-a402-01471b9863d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:27.623925 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842326ad-630b-4e01-99b4-c68fc659c638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:27.624626 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842326ad-630b-4e01-99b4-c68fc659c638 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:27.624904 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842326ad-630b-4e01-99b4-c68fc659c638 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:27.625171 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842326ad-630b-4e01-99b4-c68fc659c638 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:27.631365 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842326ad-630b-4e01-99b4-c68fc659c638 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:27.631365 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842326ad-630b-4e01-99b4-c68fc659c638 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:27.631365 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842326ad-630b-4e01-99b4-c68fc659c638 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:27.641241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d36a62a5-55d0-4aba-a402-01471b9863d9 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:27.642429 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d36a62a5-55d0-4aba-a402-01471b9863d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:27.647573 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 324/636] 10.222.0.48 () {56 vars in 1137 bytes} [Fri Sep 9 20:47:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:27.653011 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842326ad-630b-4e01-99b4-c68fc659c638 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:27.660668 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:27.722188 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 313/637] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:47:27.722981 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a62a5-55d0-4aba-a402-01471b9863d9 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:27.771395 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842326ad-630b-4e01-99b4-c68fc659c638 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:27.799226 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:27.812622 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851b105-a83b-4db1-99ad-8ab0d9b03f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:27.813255 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851b105-a83b-4db1-99ad-8ab0d9b03f83 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:27.813531 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851b105-a83b-4db1-99ad-8ab0d9b03f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:27.813796 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851b105-a83b-4db1-99ad-8ab0d9b03f83 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:27.824187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851b105-a83b-4db1-99ad-8ab0d9b03f83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:47:27.829939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-b851b105-a83b-4db1-99ad-8ab0d9b03f83 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:27.829939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:27.829939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:27.829939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:27.829939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:27.829939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:27.829939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:27.829939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:27.829939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:27.829939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:27.829939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:27.829939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:27.829939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:47:27.829939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:47:27.829939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:47:27.829939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:47:27.830889 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:47:27.830889 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:47:27.830889 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:47:27.830889 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:47:27.830889 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:27.830889 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:27.830889 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:27.830889 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:27.830889 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:27.830889 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:27.830889 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:27.830889 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:27.830889 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:27.830889 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:27.830889 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:27.830889 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:27.831729 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:27.831729 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:27.831729 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:27.831729 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:27.831729 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:47:27.831729 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:47:27.831729 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:47:27.831729 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:47:27.831729 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:47:27.831729 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:47:27.831729 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:27.831729 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:27.835883 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 325/638] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Sep 9 20:47:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:27.836782 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851b105-a83b-4db1-99ad-8ab0d9b03f83 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:27.852284 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a62a5-55d0-4aba-a402-01471b9863d9 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:27.902505 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a6935-6459-488b-aba0-318ffba098fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:27.903693 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a6935-6459-488b-aba0-318ffba098fc None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:27.904253 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a6935-6459-488b-aba0-318ffba098fc None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:27.905312 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a6935-6459-488b-aba0-318ffba098fc None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:27.906478 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a6935-6459-488b-aba0-318ffba098fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:27.906979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a6935-6459-488b-aba0-318ffba098fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:27.909649 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a6935-6459-488b-aba0-318ffba098fc None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:27.926255 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 314/639] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:27.927627 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a6935-6459-488b-aba0-318ffba098fc None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:27.942063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851b105-a83b-4db1-99ad-8ab0d9b03f83 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:27.946563 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:27.982888 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a560bec-f17c-4311-9c49-f113aa1e9119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:27.983652 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a560bec-f17c-4311-9c49-f113aa1e9119 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:27.983652 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a560bec-f17c-4311-9c49-f113aa1e9119 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:27.983944 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a560bec-f17c-4311-9c49-f113aa1e9119 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:28.045453 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:28.055428 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a6935-6459-488b-aba0-318ffba098fc None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:28.071837 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a560bec-f17c-4311-9c49-f113aa1e9119 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:28.073980 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a560bec-f17c-4311-9c49-f113aa1e9119 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:28.089444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1455651-34bd-41f2-9dfb-3c1efd7dad77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:28.090054 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1455651-34bd-41f2-9dfb-3c1efd7dad77 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:28.090335 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1455651-34bd-41f2-9dfb-3c1efd7dad77 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:28.090608 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1455651-34bd-41f2-9dfb-3c1efd7dad77 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:28.091283 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1455651-34bd-41f2-9dfb-3c1efd7dad77 None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:28.091591 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1455651-34bd-41f2-9dfb-3c1efd7dad77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:28.092985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1455651-34bd-41f2-9dfb-3c1efd7dad77 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:28.101392 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 315/640] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Sep 9 20:47:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:28.102221 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1455651-34bd-41f2-9dfb-3c1efd7dad77 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:28.176042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:28.190541 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 326/641] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:47:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:47:28.190541 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a560bec-f17c-4311-9c49-f113aa1e9119 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:28.206783 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1455651-34bd-41f2-9dfb-3c1efd7dad77 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:28.241325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cdf9a4-97e9-4eb5-bde2-aaa30a422efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:28.241875 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cdf9a4-97e9-4eb5-bde2-aaa30a422efa None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:28.242154 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cdf9a4-97e9-4eb5-bde2-aaa30a422efa None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:28.242423 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cdf9a4-97e9-4eb5-bde2-aaa30a422efa None admin] PATH_INFO: `/v3/users/nova` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:28.249721 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cdf9a4-97e9-4eb5-bde2-aaa30a422efa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:28.251160 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cdf9a4-97e9-4eb5-bde2-aaa30a422efa None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:28.262215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-58cdf9a4-97e9-4eb5-bde2-aaa30a422efa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:47:28.262215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:28.262215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:28.262215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:28.262215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:28.262215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:28.262215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:28.262215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:28.262215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:28.262215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:28.262215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:28.262215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:28.262215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:47:28.262215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:47:28.262215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:47:28.262215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:47:28.263147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:28.263147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:28.263147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:47:28.263147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:28.263147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:47:28.263147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:28.263147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:28.263147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:28.263147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:28.263147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:28.263147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:28.263147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:28.263147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:28.263147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:28.263147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:28.263147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:28.263147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:28.263147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:28.264614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:28.264614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:28.264614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:47:28.264614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:47:28.264614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:47:28.264614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:47:28.264614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:47:28.264614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:47:28.264614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:47:28.264614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:28.264614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 316/642] 10.222.0.48 () {56 vars in 1093 bytes} [Fri Sep 9 20:47:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:28.264614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cdf9a4-97e9-4eb5-bde2-aaa30a422efa None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:28.283255 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:28.302682 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a560bec-f17c-4311-9c49-f113aa1e9119 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:28.337315 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26abcacc-8e6b-4c54-b6f9-497cf764eda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:28.337938 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abcacc-8e6b-4c54-b6f9-497cf764eda7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:28.338214 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abcacc-8e6b-4c54-b6f9-497cf764eda7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:28.338481 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abcacc-8e6b-4c54-b6f9-497cf764eda7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:28.349168 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26abcacc-8e6b-4c54-b6f9-497cf764eda7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:47:28.354893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-26abcacc-8e6b-4c54-b6f9-497cf764eda7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:28.354893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:28.354893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:28.354893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:28.354893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:28.354893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:28.354893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:28.354893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:28.354893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:28.354893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:28.354893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:28.354893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:28.354893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:47:28.354893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:47:28.354893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:47:28.354893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:47:28.356007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:47:28.356007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:47:28.356007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:47:28.356007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:47:28.356007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:28.356007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:28.356007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:28.356007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:28.356007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:28.356007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:28.356007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:28.356007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:28.356007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:28.356007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:28.356007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:28.356007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:28.356897 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:28.356897 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:28.356897 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:28.356897 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:28.356897 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:47:28.356897 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:47:28.356897 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:47:28.356897 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:47:28.356897 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:47:28.356897 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:47:28.356897 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:28.356897 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:28.356897 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 327/643] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Sep 9 20:47:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:28.358213 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26abcacc-8e6b-4c54-b6f9-497cf764eda7 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:28.382396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cdf9a4-97e9-4eb5-bde2-aaa30a422efa None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:28.412512 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:28.417511 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c478c9-d5a3-4f6e-9627-26792eb531c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:28.418138 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c478c9-d5a3-4f6e-9627-26792eb531c9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:28.418413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c478c9-d5a3-4f6e-9627-26792eb531c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:28.418683 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c478c9-d5a3-4f6e-9627-26792eb531c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:28.429057 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c478c9-d5a3-4f6e-9627-26792eb531c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:28.430494 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c478c9-d5a3-4f6e-9627-26792eb531c9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:28.445215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-78c478c9-d5a3-4f6e-9627-26792eb531c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:28.445215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:28.445215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:28.445215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:28.445215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:28.445215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:28.445215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:28.445215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:28.445215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:28.445215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:28.445215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:28.445215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:28.445215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:28.445215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:28.445215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:28.445215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:28.446165 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:28.446165 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:28.446165 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:28.446165 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:28.446165 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:28.446165 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:28.446165 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:28.446165 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:28.446165 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:28.446165 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:28.446165 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:28.446165 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:28.446165 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:28.446165 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:28.446165 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:28.446165 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:28.446165 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:28.447671 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:28.447671 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:28.447671 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:28.447671 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 317/644] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:28.447671 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c478c9-d5a3-4f6e-9627-26792eb531c9 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:28.476683 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26abcacc-8e6b-4c54-b6f9-497cf764eda7 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:28.512580 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431edfa0-7951-4bdf-a9b8-c22982ce2544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:28.513161 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431edfa0-7951-4bdf-a9b8-c22982ce2544 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:28.513439 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431edfa0-7951-4bdf-a9b8-c22982ce2544 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:28.513730 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431edfa0-7951-4bdf-a9b8-c22982ce2544 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:28.524695 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431edfa0-7951-4bdf-a9b8-c22982ce2544 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:28.526116 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431edfa0-7951-4bdf-a9b8-c22982ce2544 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:28.527932 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:28.541449 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-431edfa0-7951-4bdf-a9b8-c22982ce2544 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:28.541449 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:28.541449 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:28.541449 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:28.541449 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:28.541449 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:28.541449 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:28.541449 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:28.541449 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:28.541449 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:28.541449 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:28.541449 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:28.541449 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:28.541449 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:28.541449 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:28.541449 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:28.542594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:28.542594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:28.542594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:28.542594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:28.542594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:28.542594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:28.542594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:28.542594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:28.542594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:28.542594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:28.542594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:28.542594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:28.542594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:28.542594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:28.542594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:28.542594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:28.542594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:28.543491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:28.543491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:28.543491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:28.543491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 328/645] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:28.543724 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431edfa0-7951-4bdf-a9b8-c22982ce2544 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:28.555606 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c478c9-d5a3-4f6e-9627-26792eb531c9 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:28.592293 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44581c-9d5a-4b0e-87d0-30747b27096d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:28.595227 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44581c-9d5a-4b0e-87d0-30747b27096d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:28.595227 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44581c-9d5a-4b0e-87d0-30747b27096d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:28.595227 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44581c-9d5a-4b0e-87d0-30747b27096d None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:28.595227 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb44581c-9d5a-4b0e-87d0-30747b27096d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:28.595227 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb44581c-9d5a-4b0e-87d0-30747b27096d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:28.596055 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb44581c-9d5a-4b0e-87d0-30747b27096d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:28.617998 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 318/646] 10.222.0.48 () {56 vars in 1133 bytes} [Fri Sep 9 20:47:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:28.617998 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44581c-9d5a-4b0e-87d0-30747b27096d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:28.656239 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431edfa0-7951-4bdf-a9b8-c22982ce2544 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:28.662006 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:28.690514 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c3499-5ef9-4e45-b4c0-9c95a94d59f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:28.691122 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c3499-5ef9-4e45-b4c0-9c95a94d59f8 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:28.691400 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c3499-5ef9-4e45-b4c0-9c95a94d59f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:28.691666 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c3499-5ef9-4e45-b4c0-9c95a94d59f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:28.692352 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c3499-5ef9-4e45-b4c0-9c95a94d59f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:28.692653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c3499-5ef9-4e45-b4c0-9c95a94d59f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:28.694688 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c3499-5ef9-4e45-b4c0-9c95a94d59f8 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:28.703119 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 329/647] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Sep 9 20:47:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:28.708498 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c3499-5ef9-4e45-b4c0-9c95a94d59f8 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:28.719131 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44581c-9d5a-4b0e-87d0-30747b27096d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:28.752876 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af667c65-bc59-401e-81b5-66e2f583993a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:28.753517 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af667c65-bc59-401e-81b5-66e2f583993a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:28.753761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af667c65-bc59-401e-81b5-66e2f583993a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:28.754029 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af667c65-bc59-401e-81b5-66e2f583993a None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:28.754665 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af667c65-bc59-401e-81b5-66e2f583993a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:28.754966 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af667c65-bc59-401e-81b5-66e2f583993a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:28.756375 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af667c65-bc59-401e-81b5-66e2f583993a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:28.767556 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:28.768063 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 319/648] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:28.773261 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af667c65-bc59-401e-81b5-66e2f583993a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:28.819974 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c3499-5ef9-4e45-b4c0-9c95a94d59f8 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:28.857114 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949413e8-9f23-49b4-8a6d-6f062e17a926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:28.858453 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949413e8-9f23-49b4-8a6d-6f062e17a926 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:28.858952 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949413e8-9f23-49b4-8a6d-6f062e17a926 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:28.859227 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949413e8-9f23-49b4-8a6d-6f062e17a926 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:28.859879 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949413e8-9f23-49b4-8a6d-6f062e17a926 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:28.860266 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949413e8-9f23-49b4-8a6d-6f062e17a926 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:28.861640 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949413e8-9f23-49b4-8a6d-6f062e17a926 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:28.878456 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 330/649] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:28.878456 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949413e8-9f23-49b4-8a6d-6f062e17a926 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:28.885886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af667c65-bc59-401e-81b5-66e2f583993a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:28.887446 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:28.920895 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789b676-3dcf-405f-b530-ff766e644ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:28.921500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789b676-3dcf-405f-b530-ff766e644ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:28.921776 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789b676-3dcf-405f-b530-ff766e644ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:28.922041 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789b676-3dcf-405f-b530-ff766e644ce3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:28.986250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949413e8-9f23-49b4-8a6d-6f062e17a926 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:28.998677 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d789b676-3dcf-405f-b530-ff766e644ce3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:29.000183 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d789b676-3dcf-405f-b530-ff766e644ce3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:29.003684 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:29.025768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc35cfb1-366e-4600-ac94-20ebb588e70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:29.026489 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35cfb1-366e-4600-ac94-20ebb588e70c None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:29.026835 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35cfb1-366e-4600-ac94-20ebb588e70c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:29.027109 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35cfb1-366e-4600-ac94-20ebb588e70c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:29.037825 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc35cfb1-366e-4600-ac94-20ebb588e70c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:29.039818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc35cfb1-366e-4600-ac94-20ebb588e70c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:29.058033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-cc35cfb1-366e-4600-ac94-20ebb588e70c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:29.058033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:29.058033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:29.058033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:29.058033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:29.058033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:29.058033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:29.058033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:29.058033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:29.058033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:29.058033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:29.058033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:29.058033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:29.058033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:29.058033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:29.058033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:29.058980 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:29.058980 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:29.058980 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:29.058980 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:29.058980 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:29.058980 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:29.058980 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:29.058980 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:29.058980 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:29.058980 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:29.058980 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:29.058980 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:29.058980 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:29.058980 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:29.058980 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:29.058980 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:29.058980 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:29.059865 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:29.059865 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:29.059865 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:29.059865 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 331/650] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:29.060806 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc35cfb1-366e-4600-ac94-20ebb588e70c None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:29.113780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 320/651] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:47:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:47:29.114981 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789b676-3dcf-405f-b530-ff766e644ce3 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:29.118332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:29.178275 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc35cfb1-366e-4600-ac94-20ebb588e70c None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:29.213549 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c914ec38-496f-4c36-89ac-9887fe84f449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:29.214324 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914ec38-496f-4c36-89ac-9887fe84f449 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:29.214612 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914ec38-496f-4c36-89ac-9887fe84f449 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:29.214891 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914ec38-496f-4c36-89ac-9887fe84f449 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:29.223094 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c914ec38-496f-4c36-89ac-9887fe84f449 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:29.223900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789b676-3dcf-405f-b530-ff766e644ce3 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:29.224754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c914ec38-496f-4c36-89ac-9887fe84f449 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:29.236451 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-c914ec38-496f-4c36-89ac-9887fe84f449 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 20:47:29.236451 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:29.236451 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:29.236451 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:29.236451 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:29.236451 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:29.236451 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:29.236451 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:29.236451 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:29.236451 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:29.236451 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:29.236451 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:29.236451 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:47:29.236451 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:47:29.236451 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:47:29.236451 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:47:29.237561 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:29.237561 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:29.237561 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:47:29.237561 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:29.237561 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:47:29.237561 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:29.237561 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:29.237561 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:29.237561 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:29.237561 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:29.237561 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:29.237561 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:29.237561 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:29.237561 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:29.237561 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:29.237561 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:29.237561 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:29.237561 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:29.238679 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:29.238679 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:29.238679 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:47:29.238679 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:47:29.238679 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:47:29.238679 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:47:29.238679 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:47:29.238679 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:47:29.238679 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 20:47:29.238679 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:29.238679 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 332/652] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:47:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:29.239262 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c914ec38-496f-4c36-89ac-9887fe84f449 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:29.262932 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a13372b-a3af-46c2-ad0f-7094f39bdbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:29.263562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13372b-a3af-46c2-ad0f-7094f39bdbbb None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:29.263850 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13372b-a3af-46c2-ad0f-7094f39bdbbb None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:29.264157 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13372b-a3af-46c2-ad0f-7094f39bdbbb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:29.267048 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:29.270748 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a13372b-a3af-46c2-ad0f-7094f39bdbbb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:29.272288 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a13372b-a3af-46c2-ad0f-7094f39bdbbb None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:29.283413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-0a13372b-a3af-46c2-ad0f-7094f39bdbbb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:29.283413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:29.283413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:29.283413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:29.283413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:29.283413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:29.283413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:29.283413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:29.283413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:29.283413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:29.283413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:29.283413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:29.283413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:47:29.283413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:47:29.283413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:47:29.284499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:47:29.284499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:29.284499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:29.284499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:29.284499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:29.284499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:29.284499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:29.284499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:29.284499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:29.284499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:29.284499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:29.284499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:29.284499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:29.284499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:29.284499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:29.284499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:29.284499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:29.285413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:47:29.285413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:47:29.285413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:47:29.285413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:47:29.285413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:47:29.285413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:47:29.285413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:29.285413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:29.285413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 321/653] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:47:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:29.285413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a13372b-a3af-46c2-ad0f-7094f39bdbbb None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:29.361850 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c914ec38-496f-4c36-89ac-9887fe84f449 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:29.370826 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:29.399318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8281d-4501-4502-926d-d069436fdc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:29.399933 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8281d-4501-4502-926d-d069436fdc5b None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:29.400256 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8281d-4501-4502-926d-d069436fdc5b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:29.400527 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8281d-4501-4502-926d-d069436fdc5b None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:29.401166 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d8281d-4501-4502-926d-d069436fdc5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:29.401466 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d8281d-4501-4502-926d-d069436fdc5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:29.402837 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d8281d-4501-4502-926d-d069436fdc5b None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:29.408586 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a13372b-a3af-46c2-ad0f-7094f39bdbbb None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:29.414571 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 333/654] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:29.415749 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b91c6-adb7-41f1-b55a-c20f9ecad486 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:29.416033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b91c6-adb7-41f1-b55a-c20f9ecad486 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:29.416342 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b91c6-adb7-41f1-b55a-c20f9ecad486 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:29.417533 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 334/655] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:29.418229 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b91c6-adb7-41f1-b55a-c20f9ecad486 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:29.446627 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f49e65-5ff9-493c-bdc5-165d05bb0bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:29.447259 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f49e65-5ff9-493c-bdc5-165d05bb0bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:29.447536 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f49e65-5ff9-493c-bdc5-165d05bb0bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:29.447811 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f49e65-5ff9-493c-bdc5-165d05bb0bf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:29.457474 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f49e65-5ff9-493c-bdc5-165d05bb0bf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:29.458906 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f49e65-5ff9-493c-bdc5-165d05bb0bf6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:29.474089 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-a5f49e65-5ff9-493c-bdc5-165d05bb0bf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:29.474089 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:29.474089 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:29.474089 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:29.474089 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:29.474089 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:29.474089 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:29.474089 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:29.474089 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:29.474089 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:29.474089 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:29.474089 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:29.474089 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:29.474089 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:29.474089 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:29.474089 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:29.475001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:29.475001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:29.475001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:29.475001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:29.475001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:29.475001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:29.475001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:29.475001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:29.475001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:29.475001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:29.475001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:29.475001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:29.475001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:29.475001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:29.475001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:29.475001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:29.475001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:29.476754 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:29.476754 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:29.476754 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:29.476754 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 322/656] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:29.476995 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f49e65-5ff9-493c-bdc5-165d05bb0bf6 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:29.509228 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:29.523985 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b91c6-adb7-41f1-b55a-c20f9ecad486 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:29.557782 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a651fc2-7d5e-45c7-9cce-af927dbace74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:29.558321 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a651fc2-7d5e-45c7-9cce-af927dbace74 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:29.558596 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a651fc2-7d5e-45c7-9cce-af927dbace74 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:29.558865 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a651fc2-7d5e-45c7-9cce-af927dbace74 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:29.559578 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a651fc2-7d5e-45c7-9cce-af927dbace74 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:29.559879 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a651fc2-7d5e-45c7-9cce-af927dbace74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:29.561594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a651fc2-7d5e-45c7-9cce-af927dbace74 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:29.573690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f49e65-5ff9-493c-bdc5-165d05bb0bf6 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:29.578146 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 335/657] 10.222.0.48 () {56 vars in 1137 bytes} [Fri Sep 9 20:47:29 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:29.578942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a651fc2-7d5e-45c7-9cce-af927dbace74 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:29.587665 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:29.607418 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbbb5c8-a3d2-4b55-b34d-16cb39b620ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:29.608022 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbb5c8-a3d2-4b55-b34d-16cb39b620ea None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:29.608326 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbb5c8-a3d2-4b55-b34d-16cb39b620ea None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:29.608599 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbb5c8-a3d2-4b55-b34d-16cb39b620ea None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:29.609284 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbbb5c8-a3d2-4b55-b34d-16cb39b620ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:29.609590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbbb5c8-a3d2-4b55-b34d-16cb39b620ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:29.611038 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbbb5c8-a3d2-4b55-b34d-16cb39b620ea None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:29.622606 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 323/658] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:47:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:29.623787 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd43b08-5e19-460f-95cf-add7b12fedc9 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:29.624098 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd43b08-5e19-460f-95cf-add7b12fedc9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:29.624370 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd43b08-5e19-460f-95cf-add7b12fedc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:29.644988 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-7cd43b08-5e19-460f-95cf-add7b12fedc9 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:29.646202 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7cd43b08-5e19-460f-95cf-add7b12fedc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:29.683540 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a651fc2-7d5e-45c7-9cce-af927dbace74 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:29.719819 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea3c85-b721-4759-b0c0-95035dfefb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:29.720477 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea3c85-b721-4759-b0c0-95035dfefb8f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:29.720755 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea3c85-b721-4759-b0c0-95035dfefb8f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:29.721033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea3c85-b721-4759-b0c0-95035dfefb8f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:29.729182 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ea3c85-b721-4759-b0c0-95035dfefb8f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:29.729656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 324/659] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:47:29.730499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd43b08-5e19-460f-95cf-add7b12fedc9 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:29.730614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ea3c85-b721-4759-b0c0-95035dfefb8f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:29.752982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-26ea3c85-b721-4759-b0c0-95035dfefb8f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 20:47:29.752982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:29.752982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:29.752982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:29.752982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:29.752982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:29.752982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:29.752982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:29.752982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:29.752982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:29.752982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:29.752982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:29.752982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:47:29.752982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:47:29.752982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:47:29.752982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:47:29.754195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:29.754195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:29.754195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:47:29.754195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:29.754195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:47:29.754195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:29.754195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:29.754195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:29.754195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:29.754195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:29.754195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:29.754195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:29.754195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:29.754195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:29.754195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:29.754195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:29.754195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:29.754195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:29.755495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:29.755495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:29.755495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:47:29.755495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:47:29.755495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:47:29.755495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:47:29.755495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:47:29.755495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:47:29.755495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 20:47:29.755495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:29.758931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 336/660] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Sep 9 20:47:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:29.760238 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea3c85-b721-4759-b0c0-95035dfefb8f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:29.770152 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:29.839454 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd43b08-5e19-460f-95cf-add7b12fedc9 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:29.841667 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:29.868892 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea3c85-b721-4759-b0c0-95035dfefb8f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:29.875600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8cea8b-06f6-4f2f-9b21-dc69c473bee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:29.876649 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cea8b-06f6-4f2f-9b21-dc69c473bee8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:29.876933 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cea8b-06f6-4f2f-9b21-dc69c473bee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:29.877203 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cea8b-06f6-4f2f-9b21-dc69c473bee8 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:29.878192 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8cea8b-06f6-4f2f-9b21-dc69c473bee8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:29.878503 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8cea8b-06f6-4f2f-9b21-dc69c473bee8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:29.880208 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8cea8b-06f6-4f2f-9b21-dc69c473bee8 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:29.896270 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 325/661] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:29.896270 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8cea8b-06f6-4f2f-9b21-dc69c473bee8 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:29.909245 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04457d-f315-40b5-a096-cb45aac0ba46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:29.910021 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04457d-f315-40b5-a096-cb45aac0ba46 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:29.910670 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04457d-f315-40b5-a096-cb45aac0ba46 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:29.910960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04457d-f315-40b5-a096-cb45aac0ba46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:29.988526 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d04457d-f315-40b5-a096-cb45aac0ba46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:29.989984 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d04457d-f315-40b5-a096-cb45aac0ba46 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:29.998758 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:30.002359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8cea8b-06f6-4f2f-9b21-dc69c473bee8 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:30.037969 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11ca43-128c-45c6-ba69-2427ae147416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:30.038573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11ca43-128c-45c6-ba69-2427ae147416 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:30.038848 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11ca43-128c-45c6-ba69-2427ae147416 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:30.039115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11ca43-128c-45c6-ba69-2427ae147416 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:30.039831 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b11ca43-128c-45c6-ba69-2427ae147416 None admin] RBAC: Adding query filter params (role.id=1e000597af164d54a40df2a39c729820, scope.project.id=91319aa782334fb9930770029b74e84a, user.id=7832e97d7e97439e821268d05816ae3d) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:30.040226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b11ca43-128c-45c6-ba69-2427ae147416 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:30.041656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b11ca43-128c-45c6-ba69-2427ae147416 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:30.047326 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 326/662] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:47:29 2022] GET /identity/v3/role_assignments?user.id=7832e97d7e97439e821268d05816ae3d&scope.project.id=91319aa782334fb9930770029b74e84a&role.id=1e000597af164d54a40df2a39c729820 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:30.048379 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11ca43-128c-45c6-ba69-2427ae147416 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:30.081212 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:30.102370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 337/663] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:47:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:47:30.102370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04457d-f315-40b5-a096-cb45aac0ba46 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:30.165684 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11ca43-128c-45c6-ba69-2427ae147416 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:30.203729 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d900dd8-1a9b-4330-bd3e-a125a2a56295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:30.204371 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d900dd8-1a9b-4330-bd3e-a125a2a56295 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:30.204650 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d900dd8-1a9b-4330-bd3e-a125a2a56295 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:30.204924 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d900dd8-1a9b-4330-bd3e-a125a2a56295 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:30.205667 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d900dd8-1a9b-4330-bd3e-a125a2a56295 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:30.205978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d900dd8-1a9b-4330-bd3e-a125a2a56295 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:30.207453 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d900dd8-1a9b-4330-bd3e-a125a2a56295 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:30.215793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04457d-f315-40b5-a096-cb45aac0ba46 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:30.225287 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:30.227418 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 327/664] 10.222.0.48 () {56 vars in 1139 bytes} [Fri Sep 9 20:47:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:30.228325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d900dd8-1a9b-4330-bd3e-a125a2a56295 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:30.258620 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc18ad-16dd-40dc-97fc-ec7a349b50aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:30.259888 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc18ad-16dd-40dc-97fc-ec7a349b50aa None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:30.259888 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc18ad-16dd-40dc-97fc-ec7a349b50aa None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:30.260418 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc18ad-16dd-40dc-97fc-ec7a349b50aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:30.268718 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc18ad-16dd-40dc-97fc-ec7a349b50aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:30.270182 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc18ad-16dd-40dc-97fc-ec7a349b50aa None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:30.282844 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-adfc18ad-16dd-40dc-97fc-ec7a349b50aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:30.282844 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:30.282844 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:30.282844 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:30.282844 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:30.282844 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:30.282844 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:30.282844 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:30.282844 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:30.282844 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:30.282844 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:30.282844 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:30.282844 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:47:30.282844 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:47:30.282844 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:47:30.283768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:47:30.283768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:30.283768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:30.283768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:30.283768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:30.283768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:30.283768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:30.283768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:30.283768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:30.283768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:30.283768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:30.283768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:30.283768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:30.283768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:30.283768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:30.283768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:30.283768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:30.284696 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:47:30.284696 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:47:30.284696 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:47:30.284696 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:47:30.284696 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:47:30.284696 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:47:30.284696 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:30.284696 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:30.284696 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 338/665] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:47:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:30.285201 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc18ad-16dd-40dc-97fc-ec7a349b50aa None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:30.331011 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:30.346363 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d900dd8-1a9b-4330-bd3e-a125a2a56295 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:30.382703 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6abc1-6e02-4adc-9871-5d4e051d99f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:30.383287 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6abc1-6e02-4adc-9871-5d4e051d99f7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:30.383537 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6abc1-6e02-4adc-9871-5d4e051d99f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:30.383808 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6abc1-6e02-4adc-9871-5d4e051d99f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:30.392439 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc18ad-16dd-40dc-97fc-ec7a349b50aa None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:30.394258 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f6abc1-6e02-4adc-9871-5d4e051d99f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:47:30.400409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-82f6abc1-6e02-4adc-9871-5d4e051d99f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:30.400409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:30.400409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:30.400409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:30.400409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:30.400409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:30.400409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:30.400409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:30.400409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:30.400409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:30.400409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:30.400409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:30.400409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:47:30.400409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:47:30.400409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:47:30.400409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:47:30.401377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:47:30.401377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:47:30.401377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:47:30.401377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:47:30.401377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:30.401377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:30.401377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:30.401377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:30.401377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:30.401377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:30.401377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:30.401377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:30.401377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:30.401377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:30.401377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:30.401377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:30.402234 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:30.402234 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:30.402234 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:30.402234 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:30.402234 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:47:30.402234 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:47:30.402234 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:47:30.402234 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:47:30.402234 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:47:30.402234 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:47:30.402234 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:30.402234 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:30.402234 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 328/666] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Sep 9 20:47:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:30.403466 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6abc1-6e02-4adc-9871-5d4e051d99f7 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:30.429445 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ce6cb-3891-44a5-9932-b3918a94da97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:30.430544 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ce6cb-3891-44a5-9932-b3918a94da97 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:30.430832 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ce6cb-3891-44a5-9932-b3918a94da97 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:30.431401 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ce6cb-3891-44a5-9932-b3918a94da97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:30.442425 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ce6cb-3891-44a5-9932-b3918a94da97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:30.443855 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ce6cb-3891-44a5-9932-b3918a94da97 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:30.458774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-925ce6cb-3891-44a5-9932-b3918a94da97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:30.458774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:30.458774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:30.458774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:30.458774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:30.458774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:30.458774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:30.458774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:30.458774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:30.458774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:30.458774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:30.458774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:30.458774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:30.458774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:30.458774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:30.458774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:30.459972 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:30.459972 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:30.459972 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:30.459972 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:30.459972 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:30.459972 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:30.459972 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:30.459972 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:30.459972 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:30.459972 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:30.459972 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:30.459972 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:30.459972 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:30.459972 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:30.459972 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:30.459972 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:30.459972 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:30.461012 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:30.461012 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:30.461012 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:30.461012 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 339/667] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:30.461012 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ce6cb-3891-44a5-9932-b3918a94da97 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:30.480864 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:30.518836 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6abc1-6e02-4adc-9871-5d4e051d99f7 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:30.556220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb98d2-757a-4913-93a9-576f87ce8ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:30.556819 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb98d2-757a-4913-93a9-576f87ce8ecf None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:30.557093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb98d2-757a-4913-93a9-576f87ce8ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:30.557367 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb98d2-757a-4913-93a9-576f87ce8ecf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:30.562120 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ce6cb-3891-44a5-9932-b3918a94da97 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:30.573668 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:30.600023 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2ca617-3a4c-4e42-882c-6b5a1b499480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:30.601370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ca617-3a4c-4e42-882c-6b5a1b499480 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:30.601811 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ca617-3a4c-4e42-882c-6b5a1b499480 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:30.602235 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ca617-3a4c-4e42-882c-6b5a1b499480 None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:30.603877 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2ca617-3a4c-4e42-882c-6b5a1b499480 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:30.604216 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2ca617-3a4c-4e42-882c-6b5a1b499480 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:30.605643 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2ca617-3a4c-4e42-882c-6b5a1b499480 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:30.623127 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 340/668] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:47:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:30.623127 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2ca617-3a4c-4e42-882c-6b5a1b499480 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:30.634297 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb98d2-757a-4913-93a9-576f87ce8ecf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:30.635503 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb98d2-757a-4913-93a9-576f87ce8ecf None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:30.708124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:30.733378 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2ca617-3a4c-4e42-882c-6b5a1b499480 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:30.745002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 329/669] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:47:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:47:30.745002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb98d2-757a-4913-93a9-576f87ce8ecf None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:30.775053 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b259790-6f27-4c44-9b39-5f0df38768f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:30.775770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b259790-6f27-4c44-9b39-5f0df38768f7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:30.776049 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b259790-6f27-4c44-9b39-5f0df38768f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:30.776448 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b259790-6f27-4c44-9b39-5f0df38768f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:30.777106 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b259790-6f27-4c44-9b39-5f0df38768f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:30.777408 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b259790-6f27-4c44-9b39-5f0df38768f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:30.778769 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b259790-6f27-4c44-9b39-5f0df38768f7 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:30.792092 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 341/670] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Sep 9 20:47:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:30.792092 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b259790-6f27-4c44-9b39-5f0df38768f7 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:30.793710 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:30.848731 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb98d2-757a-4913-93a9-576f87ce8ecf None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:30.883803 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270822fd-b399-4ab8-8695-4f2ad8c8649d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:30.884488 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270822fd-b399-4ab8-8695-4f2ad8c8649d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:30.884730 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270822fd-b399-4ab8-8695-4f2ad8c8649d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:30.884979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270822fd-b399-4ab8-8695-4f2ad8c8649d None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:30.885615 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270822fd-b399-4ab8-8695-4f2ad8c8649d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:30.885912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270822fd-b399-4ab8-8695-4f2ad8c8649d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:30.887275 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270822fd-b399-4ab8-8695-4f2ad8c8649d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:30.898252 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 330/671] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:30.899044 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270822fd-b399-4ab8-8695-4f2ad8c8649d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:30.907968 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b259790-6f27-4c44-9b39-5f0df38768f7 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:30.945604 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca78f73-4925-4d2b-8c43-92ba371cc5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:30.946205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca78f73-4925-4d2b-8c43-92ba371cc5f7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:30.946680 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca78f73-4925-4d2b-8c43-92ba371cc5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:30.946797 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca78f73-4925-4d2b-8c43-92ba371cc5f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:30.947450 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca78f73-4925-4d2b-8c43-92ba371cc5f7 None admin] RBAC: Adding query filter params (role.id=1e000597af164d54a40df2a39c729820, scope.project.id=91319aa782334fb9930770029b74e84a, user.id=613b845a9b4d432daf8d478fc3970708) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:30.947755 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca78f73-4925-4d2b-8c43-92ba371cc5f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:30.949249 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca78f73-4925-4d2b-8c43-92ba371cc5f7 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:30.955484 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 342/672] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:47:30 2022] GET /identity/v3/role_assignments?user.id=613b845a9b4d432daf8d478fc3970708&scope.project.id=91319aa782334fb9930770029b74e84a&role.id=1e000597af164d54a40df2a39c729820 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:30.956216 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca78f73-4925-4d2b-8c43-92ba371cc5f7 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:30.960465 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:31.034507 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270822fd-b399-4ab8-8695-4f2ad8c8649d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:31.048462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:31.075407 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec06265-742e-42ca-adba-2452a94938d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:31.076170 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec06265-742e-42ca-adba-2452a94938d0 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:31.076449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec06265-742e-42ca-adba-2452a94938d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:31.076720 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec06265-742e-42ca-adba-2452a94938d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:31.089854 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec06265-742e-42ca-adba-2452a94938d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:31.091522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec06265-742e-42ca-adba-2452a94938d0 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:31.107917 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca78f73-4925-4d2b-8c43-92ba371cc5f7 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:31.109310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-8ec06265-742e-42ca-adba-2452a94938d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:31.109310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:31.109310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:31.109310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:31.109310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:31.109310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:31.109310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:31.109310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:31.109310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:31.109310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:31.109310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:31.109310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:31.109310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:31.109310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:31.109310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:31.109310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:31.110314 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:31.110314 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:31.110314 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:31.110314 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:31.110314 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:31.110314 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:31.110314 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:31.110314 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:31.110314 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:31.110314 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:31.110314 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:31.110314 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:31.110314 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:31.110314 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:31.110314 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:31.110314 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:31.110314 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:31.111208 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:31.111208 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:31.111208 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:31.111208 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 331/673] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:31.111856 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec06265-742e-42ca-adba-2452a94938d0 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:31.148954 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cefe68-8251-45f2-a2c5-d8ea3b59882f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:31.149945 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cefe68-8251-45f2-a2c5-d8ea3b59882f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:31.150432 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cefe68-8251-45f2-a2c5-d8ea3b59882f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:31.150934 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cefe68-8251-45f2-a2c5-d8ea3b59882f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:31.163975 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cefe68-8251-45f2-a2c5-d8ea3b59882f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:31.165768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cefe68-8251-45f2-a2c5-d8ea3b59882f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:31.180419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-23cefe68-8251-45f2-a2c5-d8ea3b59882f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:31.180419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:31.180419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:31.180419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:31.180419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:31.180419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:31.180419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:31.180419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:31.180419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:31.180419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:31.180419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:31.180419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:31.180419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:31.180419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:31.180419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:31.180419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:31.181403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:31.181403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:31.181403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:31.181403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:31.181403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:31.181403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:31.181403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:31.181403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:31.181403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:31.181403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:31.181403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:31.181403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:31.181403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:31.181403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:31.181403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:31.181403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:31.181403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:31.182293 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:31.182293 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:31.182293 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:31.184380 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 343/674] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:31.185655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cefe68-8251-45f2-a2c5-d8ea3b59882f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:31.222295 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec06265-742e-42ca-adba-2452a94938d0 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:31.251414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:31.267801 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5186bb-cf11-424f-a24c-1fa1a09809f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:31.268475 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5186bb-cf11-424f-a24c-1fa1a09809f5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:31.268756 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5186bb-cf11-424f-a24c-1fa1a09809f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:31.269030 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5186bb-cf11-424f-a24c-1fa1a09809f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:31.277569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5186bb-cf11-424f-a24c-1fa1a09809f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:31.279204 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5186bb-cf11-424f-a24c-1fa1a09809f5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:31.292275 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-cb5186bb-cf11-424f-a24c-1fa1a09809f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:31.292275 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:31.292275 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:31.292275 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:31.292275 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:31.292275 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:31.292275 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:31.292275 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:31.292275 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:31.292275 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:31.292275 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:31.292275 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:31.292275 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:47:31.292275 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:47:31.292275 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:47:31.293168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:47:31.293168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:31.293168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:31.293168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:31.293168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:31.293168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:31.293168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:31.293168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:31.293168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:31.293168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:31.293168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:31.293168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:31.293168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:31.293168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:31.293168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:31.293168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:31.293168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:31.294640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:47:31.294640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:47:31.294640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:47:31.294640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:47:31.294640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:47:31.294640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:47:31.294640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:31.294640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:31.294640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 332/675] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:47:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:31.294640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5186bb-cf11-424f-a24c-1fa1a09809f5 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:31.308666 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cefe68-8251-45f2-a2c5-d8ea3b59882f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:31.319428 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:31.356809 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd57eb4-1b77-41ab-9eee-3eeea285b6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:31.357835 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd57eb4-1b77-41ab-9eee-3eeea285b6d5 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:31.358474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd57eb4-1b77-41ab-9eee-3eeea285b6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:31.358967 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd57eb4-1b77-41ab-9eee-3eeea285b6d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:31.359876 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd57eb4-1b77-41ab-9eee-3eeea285b6d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:31.360468 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd57eb4-1b77-41ab-9eee-3eeea285b6d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:31.362873 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd57eb4-1b77-41ab-9eee-3eeea285b6d5 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:31.378958 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 344/676] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:31.380102 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd57eb4-1b77-41ab-9eee-3eeea285b6d5 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:31.410099 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5186bb-cf11-424f-a24c-1fa1a09809f5 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:31.445796 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b8e8d-bdc4-4c8e-bb7d-2cc952d9fda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:31.446563 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b8e8d-bdc4-4c8e-bb7d-2cc952d9fda2 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:31.446940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b8e8d-bdc4-4c8e-bb7d-2cc952d9fda2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:31.447307 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b8e8d-bdc4-4c8e-bb7d-2cc952d9fda2 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:31.448048 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b8e8d-bdc4-4c8e-bb7d-2cc952d9fda2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:31.448539 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b8e8d-bdc4-4c8e-bb7d-2cc952d9fda2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:31.450063 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b8e8d-bdc4-4c8e-bb7d-2cc952d9fda2 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:31.463329 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 333/677] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:31.464383 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b8e8d-bdc4-4c8e-bb7d-2cc952d9fda2 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:31.505211 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd57eb4-1b77-41ab-9eee-3eeea285b6d5 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:31.526321 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:31.546006 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:31.549874 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db5a013-d703-4fbe-8c2b-c1f3db4e5cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:31.550653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5a013-d703-4fbe-8c2b-c1f3db4e5cf2 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:31.551081 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5a013-d703-4fbe-8c2b-c1f3db4e5cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:31.551503 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5a013-d703-4fbe-8c2b-c1f3db4e5cf2 None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:31.552562 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db5a013-d703-4fbe-8c2b-c1f3db4e5cf2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:31.552876 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db5a013-d703-4fbe-8c2b-c1f3db4e5cf2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:31.554647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db5a013-d703-4fbe-8c2b-c1f3db4e5cf2 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:31.572757 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b8e8d-bdc4-4c8e-bb7d-2cc952d9fda2 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:31.576281 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 345/678] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:47:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:31.582577 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db5a013-d703-4fbe-8c2b-c1f3db4e5cf2 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:31.613233 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5accc71-a4e5-4638-8a42-cf7e6f8f0547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:31.614215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5accc71-a4e5-4638-8a42-cf7e6f8f0547 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:31.614693 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5accc71-a4e5-4638-8a42-cf7e6f8f0547 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:31.615251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5accc71-a4e5-4638-8a42-cf7e6f8f0547 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:31.623074 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5accc71-a4e5-4638-8a42-cf7e6f8f0547 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:31.624898 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5accc71-a4e5-4638-8a42-cf7e6f8f0547 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:31.637363 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-a5accc71-a4e5-4638-8a42-cf7e6f8f0547 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:31.637363 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:31.637363 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:31.637363 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:31.637363 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:31.637363 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:31.637363 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:31.637363 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:31.637363 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:31.637363 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:31.637363 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:31.637363 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:31.637363 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:47:31.637363 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:47:31.637363 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:47:31.638720 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:47:31.638720 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:31.638720 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:31.638720 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:31.638720 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:31.638720 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:31.638720 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:31.638720 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:31.638720 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:31.638720 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:31.638720 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:31.638720 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:31.638720 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:31.638720 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:31.638720 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:31.638720 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:31.638720 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:31.639606 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:47:31.639606 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:47:31.639606 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:47:31.639606 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:47:31.639606 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:47:31.639606 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:47:31.639606 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:31.639606 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:31.641829 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 334/679] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:47:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:31.642884 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5accc71-a4e5-4638-8a42-cf7e6f8f0547 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:31.727795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db5a013-d703-4fbe-8c2b-c1f3db4e5cf2 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:31.754079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5accc71-a4e5-4638-8a42-cf7e6f8f0547 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:31.770524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4612a8-df6c-40d9-b234-5c1f76e8b49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:31.771537 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4612a8-df6c-40d9-b234-5c1f76e8b49b None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:31.772157 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4612a8-df6c-40d9-b234-5c1f76e8b49b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:31.772666 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4612a8-df6c-40d9-b234-5c1f76e8b49b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:31.781705 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4612a8-df6c-40d9-b234-5c1f76e8b49b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:31.783596 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4612a8-df6c-40d9-b234-5c1f76e8b49b None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:31.795960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-2a4612a8-df6c-40d9-b234-5c1f76e8b49b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 20:47:31.795960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:31.795960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:31.795960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:31.795960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:31.795960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:31.795960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:31.795960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:31.795960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:31.795960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:31.795960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:31.795960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:31.795960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:47:31.795960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:47:31.795960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:47:31.795960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:47:31.796907 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:31.796907 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:31.796907 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:47:31.796907 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:31.796907 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:47:31.796907 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:31.796907 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:31.796907 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:31.796907 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:31.796907 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:31.796907 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:31.796907 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:31.796907 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:31.796907 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:31.796907 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:31.796907 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:31.796907 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:31.796907 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:31.797908 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:31.797908 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:31.797908 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:47:31.797908 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:47:31.797908 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:47:31.797908 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:47:31.797908 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:47:31.797908 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:47:31.797908 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 20:47:31.797908 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:31.800469 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 346/680] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:47:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:31.801530 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4612a8-df6c-40d9-b234-5c1f76e8b49b None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:31.801646 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1f672-bbcf-4e29-b3b7-3218e6129169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:31.802350 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1f672-bbcf-4e29-b3b7-3218e6129169 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:31.802645 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1f672-bbcf-4e29-b3b7-3218e6129169 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:31.802920 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1f672-bbcf-4e29-b3b7-3218e6129169 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:31.804235 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a1f672-bbcf-4e29-b3b7-3218e6129169 None admin] RBAC: Adding query filter params (role.id=1e000597af164d54a40df2a39c729820, scope.project.id=91319aa782334fb9930770029b74e84a, user.id=cf91131ee5bb481bbb47097cad0c003a) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:31.804550 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a1f672-bbcf-4e29-b3b7-3218e6129169 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:31.805991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a1f672-bbcf-4e29-b3b7-3218e6129169 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:31.814187 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 335/681] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:47:31 2022] GET /identity/v3/role_assignments?user.id=cf91131ee5bb481bbb47097cad0c003a&scope.project.id=91319aa782334fb9930770029b74e84a&role.id=1e000597af164d54a40df2a39c729820 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:31.815337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1f672-bbcf-4e29-b3b7-3218e6129169 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:31.825251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:31.875198 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:31.986927 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1f672-bbcf-4e29-b3b7-3218e6129169 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:32.027202 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41791f58-f4d9-4741-a9b2-9af36887184c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:32.028291 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41791f58-f4d9-4741-a9b2-9af36887184c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:32.028788 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41791f58-f4d9-4741-a9b2-9af36887184c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:32.029257 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41791f58-f4d9-4741-a9b2-9af36887184c None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:32.030412 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41791f58-f4d9-4741-a9b2-9af36887184c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:32.030961 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41791f58-f4d9-4741-a9b2-9af36887184c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:32.032735 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41791f58-f4d9-4741-a9b2-9af36887184c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:32.050305 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4612a8-df6c-40d9-b234-5c1f76e8b49b None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:32.058310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 336/682] 10.222.0.48 () {56 vars in 1135 bytes} [Fri Sep 9 20:47:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:32.059904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41791f58-f4d9-4741-a9b2-9af36887184c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:32.102057 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472754a-b2b8-4874-b557-f923593ca205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:32.103563 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472754a-b2b8-4874-b557-f923593ca205 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:32.104311 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472754a-b2b8-4874-b557-f923593ca205 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:32.104757 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472754a-b2b8-4874-b557-f923593ca205 None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:32.105922 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472754a-b2b8-4874-b557-f923593ca205 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:32.106753 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472754a-b2b8-4874-b557-f923593ca205 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:32.108759 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472754a-b2b8-4874-b557-f923593ca205 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:32.128101 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 347/683] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:47:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:32.131343 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472754a-b2b8-4874-b557-f923593ca205 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:32.152206 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:32.203827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41791f58-f4d9-4741-a9b2-9af36887184c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:32.243988 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ca890-8e32-41f5-b223-ddf3da503c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:32.244812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ca890-8e32-41f5-b223-ddf3da503c53 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:32.245195 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ca890-8e32-41f5-b223-ddf3da503c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:32.245572 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ca890-8e32-41f5-b223-ddf3da503c53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:32.265110 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:32.274924 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472754a-b2b8-4874-b557-f923593ca205 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:32.321044 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072758c9-f7d0-420b-85d6-96d8233b6040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:32.321653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072758c9-f7d0-420b-85d6-96d8233b6040 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:32.321940 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072758c9-f7d0-420b-85d6-96d8233b6040 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:32.322203 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072758c9-f7d0-420b-85d6-96d8233b6040 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:32.323847 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072758c9-f7d0-420b-85d6-96d8233b6040 None admin] RBAC: Adding query filter params (role.id=1e000597af164d54a40df2a39c729820, scope.project.id=91319aa782334fb9930770029b74e84a, user.id=8f5dfac06ac9492a86503ba133c638a9) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:32.323847 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072758c9-f7d0-420b-85d6-96d8233b6040 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:32.325773 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072758c9-f7d0-420b-85d6-96d8233b6040 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:32.331750 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ca890-8e32-41f5-b223-ddf3da503c53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:32.333244 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ca890-8e32-41f5-b223-ddf3da503c53 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:32.335352 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 348/684] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:47:32 2022] GET /identity/v3/role_assignments?user.id=8f5dfac06ac9492a86503ba133c638a9&scope.project.id=91319aa782334fb9930770029b74e84a&role.id=1e000597af164d54a40df2a39c729820 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:47:32.336589 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b32627-24e7-46df-a377-8e560c2431b6 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:32.336873 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b32627-24e7-46df-a377-8e560c2431b6 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:32.337144 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b32627-24e7-46df-a377-8e560c2431b6 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:32.338604 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 349/685] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:32.345564 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b9019-f047-46ba-970f-7350e77f5fcd None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:32.346145 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b9019-f047-46ba-970f-7350e77f5fcd None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:32.346420 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b9019-f047-46ba-970f-7350e77f5fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:32.369724 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-7f1b9019-f047-46ba-970f-7350e77f5fcd None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:32.371898 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-7f1b9019-f047-46ba-970f-7350e77f5fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:32.407164 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:32.462348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 337/686] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:47:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:47:32.468756 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ca890-8e32-41f5-b223-ddf3da503c53 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:32.486167 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 350/687] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:47:32.604638 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ca890-8e32-41f5-b223-ddf3da503c53 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:32.626240 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252ec4e3-d743-4915-9e0c-326ec5d9a874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:32.626878 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ec4e3-d743-4915-9e0c-326ec5d9a874 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:32.627162 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ec4e3-d743-4915-9e0c-326ec5d9a874 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:32.627437 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ec4e3-d743-4915-9e0c-326ec5d9a874 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:32.638695 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252ec4e3-d743-4915-9e0c-326ec5d9a874 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:32.640331 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252ec4e3-d743-4915-9e0c-326ec5d9a874 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:32.655685 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-252ec4e3-d743-4915-9e0c-326ec5d9a874 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:32.655685 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:32.655685 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:32.655685 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:32.655685 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:32.655685 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:32.655685 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:32.655685 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:32.655685 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:32.655685 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:32.655685 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:32.655685 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:32.655685 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:32.655685 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:32.655685 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:32.655685 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:32.656680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:32.656680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:32.656680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:32.656680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:32.656680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:32.656680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:32.656680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:32.656680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:32.656680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:32.656680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:32.656680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:32.656680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:32.656680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:32.656680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:32.656680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:32.656680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:32.656680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:32.657594 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:32.657594 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:32.657594 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:32.657594 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 338/688] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:32.662491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b9019-f047-46ba-970f-7350e77f5fcd None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:32.760487 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:32.769941 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b9019-f047-46ba-970f-7350e77f5fcd None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:32.808499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bedd69-ceab-428b-8129-d29fa02db215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:32.809114 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bedd69-ceab-428b-8129-d29fa02db215 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:32.809397 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bedd69-ceab-428b-8129-d29fa02db215 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:32.809671 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bedd69-ceab-428b-8129-d29fa02db215 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:32.810317 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bedd69-ceab-428b-8129-d29fa02db215 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:32.810886 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bedd69-ceab-428b-8129-d29fa02db215 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:32.812284 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bedd69-ceab-428b-8129-d29fa02db215 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:32.824250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 351/689] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:47:32.831611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bedd69-ceab-428b-8129-d29fa02db215 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:32.831841 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252ec4e3-d743-4915-9e0c-326ec5d9a874 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:32.869833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:32.986055 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bedd69-ceab-428b-8129-d29fa02db215 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:33.021203 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d52c2b-9e5c-4ab1-854b-f836e8f57ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:33.021944 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d52c2b-9e5c-4ab1-854b-f836e8f57ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:33.022890 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d52c2b-9e5c-4ab1-854b-f836e8f57ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:33.023254 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d52c2b-9e5c-4ab1-854b-f836e8f57ee4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:33.030475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d52c2b-9e5c-4ab1-854b-f836e8f57ee4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:33.032051 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d52c2b-9e5c-4ab1-854b-f836e8f57ee4 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:33.039889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252ec4e3-d743-4915-9e0c-326ec5d9a874 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:33.043882 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-c3d52c2b-9e5c-4ab1-854b-f836e8f57ee4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:33.043882 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:33.043882 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:33.043882 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:33.043882 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:33.043882 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:33.043882 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:33.043882 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:33.043882 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:33.043882 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:33.043882 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:33.043882 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:33.043882 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:47:33.043882 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:47:33.043882 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:47:33.044979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:47:33.044979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:33.044979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:33.044979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:33.044979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:33.044979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:33.044979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:33.044979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:33.044979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:33.044979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:33.044979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:33.044979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:33.044979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:33.044979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:33.044979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:33.044979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:33.044979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:33.046111 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:47:33.046111 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:47:33.046111 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:47:33.046111 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:47:33.046111 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:47:33.046111 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:47:33.046111 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:33.046111 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:33.052091 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 352/690] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:47:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:33.053335 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d52c2b-9e5c-4ab1-854b-f836e8f57ee4 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:33.067269 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:33.105683 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a093a-fb0b-4f40-ad9e-ba135b584dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:33.106399 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a093a-fb0b-4f40-ad9e-ba135b584dab None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:33.106719 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a093a-fb0b-4f40-ad9e-ba135b584dab None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:33.107062 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a093a-fb0b-4f40-ad9e-ba135b584dab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:33.126937 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55a093a-fb0b-4f40-ad9e-ba135b584dab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:33.128554 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55a093a-fb0b-4f40-ad9e-ba135b584dab None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:33.155782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-e55a093a-fb0b-4f40-ad9e-ba135b584dab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:33.155782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:33.155782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:33.155782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:33.155782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:33.155782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:33.155782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:33.155782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:33.155782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:33.155782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:33.155782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:33.155782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:33.155782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:33.155782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:33.155782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:33.155782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:33.156820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:33.156820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:33.156820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:33.156820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:33.156820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:33.156820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:33.156820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:33.156820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:33.156820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:33.156820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:33.156820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:33.156820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:33.156820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:33.156820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:33.156820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:33.156820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:33.156820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:33.157833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:33.157833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:33.157833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:33.157833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 339/691] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:33.160974 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d52c2b-9e5c-4ab1-854b-f836e8f57ee4 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:33.164768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a093a-fb0b-4f40-ad9e-ba135b584dab None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:33.195986 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b8cc03-9af5-4aa5-807d-fcc2be2948d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:33.196638 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8cc03-9af5-4aa5-807d-fcc2be2948d2 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:33.196961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8cc03-9af5-4aa5-807d-fcc2be2948d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:33.197232 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8cc03-9af5-4aa5-807d-fcc2be2948d2 None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:33.197922 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b8cc03-9af5-4aa5-807d-fcc2be2948d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:33.198222 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b8cc03-9af5-4aa5-807d-fcc2be2948d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:33.199646 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b8cc03-9af5-4aa5-807d-fcc2be2948d2 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:33.225887 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 353/692] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:47:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:33.229949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b8cc03-9af5-4aa5-807d-fcc2be2948d2 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:33.288474 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:33.364458 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:33.374752 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b8cc03-9af5-4aa5-807d-fcc2be2948d2 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:33.382962 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a093a-fb0b-4f40-ad9e-ba135b584dab None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:33.410089 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02cd73-71f8-42e5-82f5-ae7befa79f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:33.410799 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02cd73-71f8-42e5-82f5-ae7befa79f1c None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:33.411084 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02cd73-71f8-42e5-82f5-ae7befa79f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:33.411358 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02cd73-71f8-42e5-82f5-ae7befa79f1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:33.412092 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f02cd73-71f8-42e5-82f5-ae7befa79f1c None admin] RBAC: Adding query filter params (role.id=1e000597af164d54a40df2a39c729820, scope.project.id=91319aa782334fb9930770029b74e84a, user.id=7d28fa5d49d44120bd5eaec2d4e5fdbd) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:33.412419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f02cd73-71f8-42e5-82f5-ae7befa79f1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:33.413897 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f02cd73-71f8-42e5-82f5-ae7befa79f1c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:33.424184 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 354/693] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:47:33 2022] GET /identity/v3/role_assignments?user.id=7d28fa5d49d44120bd5eaec2d4e5fdbd&scope.project.id=91319aa782334fb9930770029b74e84a&role.id=1e000597af164d54a40df2a39c729820 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:33.473819 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5668d6-c8ee-4b24-927b-236cdce5d5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:33.474570 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5668d6-c8ee-4b24-927b-236cdce5d5bb None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:33.474858 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5668d6-c8ee-4b24-927b-236cdce5d5bb None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:33.475138 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5668d6-c8ee-4b24-927b-236cdce5d5bb None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:33.475851 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5668d6-c8ee-4b24-927b-236cdce5d5bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:33.476184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5668d6-c8ee-4b24-927b-236cdce5d5bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:33.477624 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5668d6-c8ee-4b24-927b-236cdce5d5bb None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:33.497339 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 340/694] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:33.503018 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02cd73-71f8-42e5-82f5-ae7befa79f1c None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:33.612719 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2a830-89c9-41a1-8c21-6560b8cef79c None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:33.613279 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2a830-89c9-41a1-8c21-6560b8cef79c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:33.613279 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2a830-89c9-41a1-8c21-6560b8cef79c None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:33.614848 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 341/695] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:33.621237 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2a34f-92d3-47a8-a6a1-2594d377750d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:33.621537 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2a34f-92d3-47a8-a6a1-2594d377750d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:33.621795 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2a34f-92d3-47a8-a6a1-2594d377750d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:33.651996 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-35f2a34f-92d3-47a8-a6a1-2594d377750d None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:33.653266 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-35f2a34f-92d3-47a8-a6a1-2594d377750d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:33.689716 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02cd73-71f8-42e5-82f5-ae7befa79f1c None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:33.732461 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951ac06-529c-4721-9974-47ab712ec198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:33.733151 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951ac06-529c-4721-9974-47ab712ec198 None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:33.733435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951ac06-529c-4721-9974-47ab712ec198 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:33.733729 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951ac06-529c-4721-9974-47ab712ec198 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:33.735398 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e951ac06-529c-4721-9974-47ab712ec198 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:33.736799 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e951ac06-529c-4721-9974-47ab712ec198 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:33.747538 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:33.749963 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 342/696] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:47:33.855096 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 355/697] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Sep 9 20:47:33 2022] POST /identity/v3/users => generated 254 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:47:34.063382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2a34f-92d3-47a8-a6a1-2594d377750d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:34.119644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:34.167497 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2a34f-92d3-47a8-a6a1-2594d377750d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:34.202907 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80065ef-3685-4dbb-b14e-966c84b5f7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:34.203546 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80065ef-3685-4dbb-b14e-966c84b5f7c3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:34.203825 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80065ef-3685-4dbb-b14e-966c84b5f7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:34.204115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80065ef-3685-4dbb-b14e-966c84b5f7c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:34.215391 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80065ef-3685-4dbb-b14e-966c84b5f7c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:47:34.220305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-a80065ef-3685-4dbb-b14e-966c84b5f7c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:47:34.220305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:34.220305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:34.220305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:34.220305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:34.220305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:34.220305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:34.220305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:34.220305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:34.220305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:34.220305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:34.220305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:34.220305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:47:34.220305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:47:34.220305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:47:34.220305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:47:34.221236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:47:34.221236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:47:34.221236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:47:34.221236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:47:34.221236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:34.221236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:34.221236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:34.221236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:34.221236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:34.221236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:34.221236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:34.221236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:34.221236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:34.221236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:34.221236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:34.221236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:34.222444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:34.222444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:34.222444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:34.222444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:34.222444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:47:34.222444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:47:34.222444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:47:34.222444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:47:34.222444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:47:34.222444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:47:34.222444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:47:34.222444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:34.222444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 343/698] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:47:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:34.228138 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951ac06-529c-4721-9974-47ab712ec198 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:34.343974 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951ac06-529c-4721-9974-47ab712ec198 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:34.381219 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b3039-cf81-40f2-b3cb-589fd1b51af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:34.382376 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b3039-cf81-40f2-b3cb-589fd1b51af2 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:34.382651 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b3039-cf81-40f2-b3cb-589fd1b51af2 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:34.382976 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b3039-cf81-40f2-b3cb-589fd1b51af2 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:34.383655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197b3039-cf81-40f2-b3cb-589fd1b51af2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:34.383961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197b3039-cf81-40f2-b3cb-589fd1b51af2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:34.385366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197b3039-cf81-40f2-b3cb-589fd1b51af2 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:34.395024 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 356/699] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:47:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:34.401557 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80065ef-3685-4dbb-b14e-966c84b5f7c3 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:34.514279 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80065ef-3685-4dbb-b14e-966c84b5f7c3 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:34.540912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:34.566742 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecadfa2-c4d8-4ace-82bf-a2230783b359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:34.567453 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecadfa2-c4d8-4ace-82bf-a2230783b359 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:34.567740 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecadfa2-c4d8-4ace-82bf-a2230783b359 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:34.568014 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecadfa2-c4d8-4ace-82bf-a2230783b359 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:34.585734 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecadfa2-c4d8-4ace-82bf-a2230783b359 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:34.587310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecadfa2-c4d8-4ace-82bf-a2230783b359 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:34.600955 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0dc590-6d20-4ef9-bcf0-bf537e6caa04 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:34.601187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0dc590-6d20-4ef9-bcf0-bf537e6caa04 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:34.601642 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0dc590-6d20-4ef9-bcf0-bf537e6caa04 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:34.603355 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 357/700] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:34.604054 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-6ecadfa2-c4d8-4ace-82bf-a2230783b359 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:34.604054 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:34.604054 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:34.604054 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:34.604054 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:34.604054 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:34.604054 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:34.604054 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:34.604054 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:34.604054 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:34.604054 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:34.604054 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:34.604054 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:34.604054 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:34.604054 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:34.604054 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:34.605149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:34.605149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:34.605149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:34.605149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:34.605149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:34.605149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:34.605149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:34.605149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:34.605149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:34.605149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:34.605149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:34.605149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:34.605149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:34.605149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:34.605149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:34.605149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:34.605149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:34.606032 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:34.606032 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:34.606032 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:34.608568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910c41d-eb3b-479e-82c2-5c8c62c73163 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:34.608849 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910c41d-eb3b-479e-82c2-5c8c62c73163 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:34.609124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910c41d-eb3b-479e-82c2-5c8c62c73163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:34.609542 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 344/701] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:34.618845 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecadfa2-c4d8-4ace-82bf-a2230783b359 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:34.643587 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-1910c41d-eb3b-479e-82c2-5c8c62c73163 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:34.644819 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-1910c41d-eb3b-479e-82c2-5c8c62c73163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:34.754578 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecadfa2-c4d8-4ace-82bf-a2230783b359 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:34.792525 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291229b-bc6e-45fb-a9b1-3b953af7e138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:34.793147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291229b-bc6e-45fb-a9b1-3b953af7e138 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:34.793426 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291229b-bc6e-45fb-a9b1-3b953af7e138 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:34.793963 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291229b-bc6e-45fb-a9b1-3b953af7e138 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:34.795276 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:34.795738 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2291229b-bc6e-45fb-a9b1-3b953af7e138 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:34.795972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2291229b-bc6e-45fb-a9b1-3b953af7e138 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:34.797420 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2291229b-bc6e-45fb-a9b1-3b953af7e138 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:34.811017 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 345/702] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:34.816455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291229b-bc6e-45fb-a9b1-3b953af7e138 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:34.845520 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 358/703] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:47:34.846825 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:34.963727 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291229b-bc6e-45fb-a9b1-3b953af7e138 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:34.998535 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1c596-4d00-4991-8e79-04ed360c39d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:34.999173 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1c596-4d00-4991-8e79-04ed360c39d7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:34.999448 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1c596-4d00-4991-8e79-04ed360c39d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:34.999773 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1c596-4d00-4991-8e79-04ed360c39d7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:35.007567 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f1c596-4d00-4991-8e79-04ed360c39d7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:35.009073 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f1c596-4d00-4991-8e79-04ed360c39d7 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:35.020957 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-48f1c596-4d00-4991-8e79-04ed360c39d7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:47:35.020957 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:35.020957 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:35.020957 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:35.020957 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:35.020957 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:35.020957 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:35.020957 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:35.020957 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:35.020957 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:35.020957 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:35.020957 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:35.020957 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:47:35.020957 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:47:35.020957 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:47:35.020957 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:47:35.022045 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:35.022045 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:35.022045 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:47:35.022045 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:35.022045 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:47:35.022045 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:35.022045 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:35.022045 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:35.022045 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:35.022045 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:35.022045 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:35.022045 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:35.022045 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:35.022045 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:35.022045 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:35.022045 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:35.022045 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:35.022045 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:35.023042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:35.023042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:35.023042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:47:35.023042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:47:35.023042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:47:35.023042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:47:35.023042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:47:35.023042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:47:35.023042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:47:35.023042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:35.023042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 346/704] 10.222.0.48 () {56 vars in 1093 bytes} [Fri Sep 9 20:47:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:47:35.027991 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910c41d-eb3b-479e-82c2-5c8c62c73163 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:35.155905 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910c41d-eb3b-479e-82c2-5c8c62c73163 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:35.199847 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b37d86-9c58-4ca3-98ce-7c271cb19e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:35.200663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b37d86-9c58-4ca3-98ce-7c271cb19e21 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:35.200959 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b37d86-9c58-4ca3-98ce-7c271cb19e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:35.201222 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b37d86-9c58-4ca3-98ce-7c271cb19e21 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:35.201972 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b37d86-9c58-4ca3-98ce-7c271cb19e21 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:35.202280 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b37d86-9c58-4ca3-98ce-7c271cb19e21 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:35.203717 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b37d86-9c58-4ca3-98ce-7c271cb19e21 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:35.211787 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1c596-4d00-4991-8e79-04ed360c39d7 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:35.230464 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 359/705] 10.222.0.48 () {56 vars in 1133 bytes} [Fri Sep 9 20:47:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:35.242090 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b37d86-9c58-4ca3-98ce-7c271cb19e21 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:35.287548 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:35.320765 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:35.341465 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1c596-4d00-4991-8e79-04ed360c39d7 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:35.373527 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b37d86-9c58-4ca3-98ce-7c271cb19e21 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:35.387262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45906be9-1b57-4ead-bd61-47692787015b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:35.388584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45906be9-1b57-4ead-bd61-47692787015b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:35.389480 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45906be9-1b57-4ead-bd61-47692787015b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:35.390103 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45906be9-1b57-4ead-bd61-47692787015b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:35.391139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45906be9-1b57-4ead-bd61-47692787015b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:35.392907 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45906be9-1b57-4ead-bd61-47692787015b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:35.408004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-45906be9-1b57-4ead-bd61-47692787015b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 20:47:35.408004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:35.408004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:47:35.408004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:47:35.408004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:47:35.408004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:47:35.408004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:47:35.408004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:47:35.408004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 20:47:35.408004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:35.408004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:47:35.408004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:35.408004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:35.408004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:35.408004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:35.408004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:35.408004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:35.409056 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:35.409056 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:35.409056 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:35.409056 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:35.409056 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:35.409056 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:35.409056 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:47:35.409056 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:47:35.409056 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:47:35.409056 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:47:35.409056 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:35.409056 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:35.409056 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:35.409056 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:35.409056 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:35.409056 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:35.409903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:35.409903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:35.409903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:35.409903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:35.409903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:35.409903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:35.409903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:35.409903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:35.409903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:35.409903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:35.409903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:47:35.409903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:47:35.409903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 20:47:35.409903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:35.414458 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 347/706] 10.222.0.48 () {56 vars in 1129 bytes} [Fri Sep 9 20:47:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:47:35.415976 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b756b9-9dc0-4bf9-b7b9-a9f5f2e4d000 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:35.416301 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b756b9-9dc0-4bf9-b7b9-a9f5f2e4d000 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:35.416573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b756b9-9dc0-4bf9-b7b9-a9f5f2e4d000 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:35.418504 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 348/707] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:35.418867 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ffd1aa-7eae-4b87-b7c7-6ad59f620837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:35.419271 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b756b9-9dc0-4bf9-b7b9-a9f5f2e4d000 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:35.420033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ffd1aa-7eae-4b87-b7c7-6ad59f620837 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:35.420534 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ffd1aa-7eae-4b87-b7c7-6ad59f620837 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:35.421007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ffd1aa-7eae-4b87-b7c7-6ad59f620837 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:35.508609 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ffd1aa-7eae-4b87-b7c7-6ad59f620837 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:35.510384 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ffd1aa-7eae-4b87-b7c7-6ad59f620837 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:35.526124 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b756b9-9dc0-4bf9-b7b9-a9f5f2e4d000 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:35.562186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:35.565786 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3318e9ac-817b-48e1-9563-d648b95a5717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:35.566632 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318e9ac-817b-48e1-9563-d648b95a5717 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:35.567079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318e9ac-817b-48e1-9563-d648b95a5717 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:35.567487 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318e9ac-817b-48e1-9563-d648b95a5717 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:35.568345 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3318e9ac-817b-48e1-9563-d648b95a5717 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:35.568845 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3318e9ac-817b-48e1-9563-d648b95a5717 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:35.570658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3318e9ac-817b-48e1-9563-d648b95a5717 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:35.580311 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 349/708] 10.222.0.48 () {56 vars in 1124 bytes} [Fri Sep 9 20:47:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:35.581930 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7ea59-8cf5-4b44-990b-0365328a050c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:35.582430 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7ea59-8cf5-4b44-990b-0365328a050c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:35.582897 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7ea59-8cf5-4b44-990b-0365328a050c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:35.604529 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-82d7ea59-8cf5-4b44-990b-0365328a050c None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:35.607050 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-82d7ea59-8cf5-4b44-990b-0365328a050c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:35.616981 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:35.636167 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 360/709] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:47:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:47:35.637332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ffd1aa-7eae-4b87-b7c7-6ad59f620837 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:35.685261 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 350/710] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:47:35.687840 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d7ea59-8cf5-4b44-990b-0365328a050c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:35.783738 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ffd1aa-7eae-4b87-b7c7-6ad59f620837 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:35.824852 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d7ea59-8cf5-4b44-990b-0365328a050c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:35.869814 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556b50e-b92c-48c2-b916-da49fff1e3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:35.870458 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556b50e-b92c-48c2-b916-da49fff1e3b6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:35.870739 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556b50e-b92c-48c2-b916-da49fff1e3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:35.871012 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556b50e-b92c-48c2-b916-da49fff1e3b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:35.873677 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07fe4c-e9ad-4f89-8135-599c08269c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:35.874290 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07fe4c-e9ad-4f89-8135-599c08269c40 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:35.874572 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07fe4c-e9ad-4f89-8135-599c08269c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:35.874844 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07fe4c-e9ad-4f89-8135-599c08269c40 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:35.875481 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba07fe4c-e9ad-4f89-8135-599c08269c40 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:35.875785 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba07fe4c-e9ad-4f89-8135-599c08269c40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:35.877173 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba07fe4c-e9ad-4f89-8135-599c08269c40 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:35.882068 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1556b50e-b92c-48c2-b916-da49fff1e3b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:35.883507 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1556b50e-b92c-48c2-b916-da49fff1e3b6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:35.894246 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 361/711] 10.222.0.48 () {56 vars in 1124 bytes} [Fri Sep 9 20:47:35 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:35.900658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-1556b50e-b92c-48c2-b916-da49fff1e3b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:35.900658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:35.900658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:35.900658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:35.900658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:35.900658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:35.900658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:35.900658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:35.900658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:35.900658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:35.900658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:35.900658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:35.900658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:35.900658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:35.900658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:35.900658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:35.901656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:35.901656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:35.901656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:35.901656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:35.901656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:35.901656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:35.901656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:35.901656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:35.901656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:35.901656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:35.901656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:35.901656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:35.901656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:35.901656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:35.901656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:35.901656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:35.901656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:35.903136 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:35.903136 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:35.903136 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:35.903136 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 351/712] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:35.903746 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:35.907370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07fe4c-e9ad-4f89-8135-599c08269c40 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:35.912415 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:36.036503 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556b50e-b92c-48c2-b916-da49fff1e3b6 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:36.067200 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07fe4c-e9ad-4f89-8135-599c08269c40 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:36.119398 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af3d52b-84e7-41b8-bd6e-f32cc39bb990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:36.120018 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3d52b-84e7-41b8-bd6e-f32cc39bb990 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:36.120326 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3d52b-84e7-41b8-bd6e-f32cc39bb990 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:36.120600 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3d52b-84e7-41b8-bd6e-f32cc39bb990 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:36.122121 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af3d52b-84e7-41b8-bd6e-f32cc39bb990 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:36.122533 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af3d52b-84e7-41b8-bd6e-f32cc39bb990 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:36.123949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af3d52b-84e7-41b8-bd6e-f32cc39bb990 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:36.138414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 362/713] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:47:36.144824 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af3d52b-84e7-41b8-bd6e-f32cc39bb990 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:36.169786 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556b50e-b92c-48c2-b916-da49fff1e3b6 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:36.214805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea903f0-d07b-4c74-8bb5-0b123ab64bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:36.215719 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea903f0-d07b-4c74-8bb5-0b123ab64bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:36.216004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea903f0-d07b-4c74-8bb5-0b123ab64bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:36.216471 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea903f0-d07b-4c74-8bb5-0b123ab64bd0 None admin] PATH_INFO: `/v3/services/network` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:36.217104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea903f0-d07b-4c74-8bb5-0b123ab64bd0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:36.218213 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:36.219043 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea903f0-d07b-4c74-8bb5-0b123ab64bd0 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:36.230349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-cea903f0-d07b-4c74-8bb5-0b123ab64bd0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 20:47:36.230349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:36.230349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:47:36.230349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:47:36.230349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:47:36.230349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:47:36.230349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:47:36.230349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:47:36.230349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 20:47:36.230349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:36.230349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:47:36.230349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:36.230349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:36.230349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:36.230349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:36.230349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:36.230349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:36.238256 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:36.238256 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:36.238256 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:36.238256 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:36.238256 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:36.238256 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:36.238256 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:47:36.238256 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:47:36.238256 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:47:36.238256 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:47:36.238256 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:36.238256 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:36.238256 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:36.238256 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:36.238256 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:36.238256 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:36.239691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:36.239691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:36.239691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:36.239691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:36.239691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:36.239691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:36.239691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:36.239691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:36.239691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:36.239691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:36.239691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:47:36.239691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:47:36.239691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 20:47:36.239691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:36.239691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 352/714] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:47:36 2022] GET /identity/v3/services/network => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:36.240779 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacf5a8-ed46-49e0-9137-db415ce4d324 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:36.240779 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacf5a8-ed46-49e0-9137-db415ce4d324 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:36.240779 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacf5a8-ed46-49e0-9137-db415ce4d324 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:36.240779 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 353/715] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:36.240779 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3bd6a9-472e-467d-a68c-2e0f577fc259 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:36.241635 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3bd6a9-472e-467d-a68c-2e0f577fc259 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:36.241796 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3bd6a9-472e-467d-a68c-2e0f577fc259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:36.260269 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1b3bd6a9-472e-467d-a68c-2e0f577fc259 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:36.261464 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1b3bd6a9-472e-467d-a68c-2e0f577fc259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:36.266166 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af3d52b-84e7-41b8-bd6e-f32cc39bb990 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:36.311898 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d6e8fc-7386-4798-99b2-9529d76ead87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:36.312521 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6e8fc-7386-4798-99b2-9529d76ead87 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:36.312803 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6e8fc-7386-4798-99b2-9529d76ead87 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:36.313075 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6e8fc-7386-4798-99b2-9529d76ead87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:36.322607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d6e8fc-7386-4798-99b2-9529d76ead87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:36.324710 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d6e8fc-7386-4798-99b2-9529d76ead87 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:36.327338 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:36.339040 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 354/716] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:47:36.342807 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-05d6e8fc-7386-4798-99b2-9529d76ead87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:36.342807 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:36.342807 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:36.342807 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:36.342807 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:36.342807 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:36.342807 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:36.342807 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:36.342807 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:36.342807 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:36.342807 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:36.342807 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:36.342807 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:47:36.342807 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:47:36.342807 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:47:36.343672 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:47:36.343672 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:36.343672 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:36.343672 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:36.343672 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:36.343672 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:36.343672 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:36.343672 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:36.343672 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:36.343672 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:36.343672 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:36.343672 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:36.343672 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:36.343672 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:36.343672 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:36.343672 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:36.343672 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:36.344682 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:47:36.344682 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:47:36.344682 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:47:36.344682 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:47:36.344682 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:47:36.344682 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:47:36.344682 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:36.344682 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:36.344682 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 363/717] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:47:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:36.344682 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d6e8fc-7386-4798-99b2-9529d76ead87 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:36.346613 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3bd6a9-472e-467d-a68c-2e0f577fc259 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:36.463310 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d6e8fc-7386-4798-99b2-9529d76ead87 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:36.489643 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:36.502681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04663f23-c207-453a-a366-e61e06d4df1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:36.503610 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04663f23-c207-453a-a366-e61e06d4df1d None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:36.503888 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04663f23-c207-453a-a366-e61e06d4df1d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:36.504296 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04663f23-c207-453a-a366-e61e06d4df1d None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:36.505210 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04663f23-c207-453a-a366-e61e06d4df1d None admin] RBAC: Adding query filter params (name=network) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:36.505517 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04663f23-c207-453a-a366-e61e06d4df1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:36.507651 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04663f23-c207-453a-a366-e61e06d4df1d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:36.512535 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 364/718] 10.222.0.48 () {56 vars in 1112 bytes} [Fri Sep 9 20:47:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:36.518159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04663f23-c207-453a-a366-e61e06d4df1d None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:36.577219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3bd6a9-472e-467d-a68c-2e0f577fc259 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:36.638327 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665c930-51f3-4c33-acfe-49080084dc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:36.638946 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665c930-51f3-4c33-acfe-49080084dc0b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:36.639232 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665c930-51f3-4c33-acfe-49080084dc0b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:36.639509 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665c930-51f3-4c33-acfe-49080084dc0b None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:36.640246 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665c930-51f3-4c33-acfe-49080084dc0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:36.644094 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665c930-51f3-4c33-acfe-49080084dc0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:36.644094 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665c930-51f3-4c33-acfe-49080084dc0b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:36.665455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 355/719] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:47:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:47:36.672112 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665c930-51f3-4c33-acfe-49080084dc0b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:36.692945 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04663f23-c207-453a-a366-e61e06d4df1d None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:36.732106 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ded7d4-27c2-405e-afe6-336be9c87875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:36.733139 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ded7d4-27c2-405e-afe6-336be9c87875 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:36.733706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ded7d4-27c2-405e-afe6-336be9c87875 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:36.734185 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ded7d4-27c2-405e-afe6-336be9c87875 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:36.735110 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ded7d4-27c2-405e-afe6-336be9c87875 None admin] RBAC: Adding query filter params (type=network) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:36.735621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ded7d4-27c2-405e-afe6-336be9c87875 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:36.737209 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ded7d4-27c2-405e-afe6-336be9c87875 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:36.742746 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:36.744301 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 365/720] 10.222.0.48 () {56 vars in 1112 bytes} [Fri Sep 9 20:47:36 2022] GET /identity/v3/services?type=network => generated 124 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:36.745475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ded7d4-27c2-405e-afe6-336be9c87875 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:36.868296 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:36.879323 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665c930-51f3-4c33-acfe-49080084dc0b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:36.950874 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08579b3-b528-4ba3-b078-fe03619343c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:36.951611 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08579b3-b528-4ba3-b078-fe03619343c2 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:36.952009 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08579b3-b528-4ba3-b078-fe03619343c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:36.952404 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08579b3-b528-4ba3-b078-fe03619343c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:36.953265 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08579b3-b528-4ba3-b078-fe03619343c2 None admin] RBAC: Adding query filter params (role.id=fcbf519fbc84458f913671a5a1d0914c, scope.project.id=91319aa782334fb9930770029b74e84a, user.id=613b845a9b4d432daf8d478fc3970708) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:36.953569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08579b3-b528-4ba3-b078-fe03619343c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:36.955028 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08579b3-b528-4ba3-b078-fe03619343c2 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:36.955822 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ded7d4-27c2-405e-afe6-336be9c87875 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:36.966501 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 356/721] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:47:36 2022] GET /identity/v3/role_assignments?user.id=613b845a9b4d432daf8d478fc3970708&scope.project.id=91319aa782334fb9930770029b74e84a&role.id=fcbf519fbc84458f913671a5a1d0914c => generated 259 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:47:36.968864 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a56ba7-0d32-433f-82ef-f395ea042ba4 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:36.969904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a56ba7-0d32-433f-82ef-f395ea042ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:36.970411 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a56ba7-0d32-433f-82ef-f395ea042ba4 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:36.973281 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 357/722] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:36.980800 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f41c6b-1043-4185-b9d9-45cbd56e76e2 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:36.981455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f41c6b-1043-4185-b9d9-45cbd56e76e2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:36.981920 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f41c6b-1043-4185-b9d9-45cbd56e76e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:37.008797 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-60f41c6b-1043-4185-b9d9-45cbd56e76e2 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:37.011028 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3cc04b-2477-4f04-a75a-2ab2241946f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:37.011748 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cc04b-2477-4f04-a75a-2ab2241946f5 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:37.012030 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cc04b-2477-4f04-a75a-2ab2241946f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:37.012345 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cc04b-2477-4f04-a75a-2ab2241946f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:37.014783 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-60f41c6b-1043-4185-b9d9-45cbd56e76e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:37.025075 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3cc04b-2477-4f04-a75a-2ab2241946f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:47:37.030588 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-fc3cc04b-2477-4f04-a75a-2ab2241946f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:47:37.030588 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:37.030588 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:37.030588 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:37.030588 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:37.030588 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:37.030588 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:37.030588 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:37.030588 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:37.030588 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:37.030588 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:37.030588 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:37.030588 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:47:37.030588 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:47:37.030588 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:47:37.030588 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:47:37.031569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:47:37.031569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:47:37.031569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:47:37.031569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:47:37.031569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:37.031569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:37.031569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:37.031569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:37.031569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:37.031569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:37.031569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:37.031569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:37.031569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:37.031569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:37.031569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:37.031569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:37.036704 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:37.036704 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:37.036704 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:37.036704 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:37.036704 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:47:37.036704 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:47:37.036704 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:47:37.036704 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:47:37.036704 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:47:37.036704 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:47:37.036704 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:47:37.036704 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:37.036704 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 366/723] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:47:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:37.040304 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3cc04b-2477-4f04-a75a-2ab2241946f5 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:37.118589 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 358/724] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 20:47:37.154620 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3cc04b-2477-4f04-a75a-2ab2241946f5 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:37.174502 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:37.198782 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71f2ee-64fe-490b-bb58-397a064e3f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:37.199733 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71f2ee-64fe-490b-bb58-397a064e3f51 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:37.200256 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71f2ee-64fe-490b-bb58-397a064e3f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:37.200727 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71f2ee-64fe-490b-bb58-397a064e3f51 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:37.201620 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd71f2ee-64fe-490b-bb58-397a064e3f51 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:37.202116 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd71f2ee-64fe-490b-bb58-397a064e3f51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:37.203775 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd71f2ee-64fe-490b-bb58-397a064e3f51 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:37.216063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 367/725] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:47:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:37.224127 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f41c6b-1043-4185-b9d9-45cbd56e76e2 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:37.296664 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:37.370889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f41c6b-1043-4185-b9d9-45cbd56e76e2 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:37.416950 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026895ce-e3af-4de4-a583-629ad05a9c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:37.417587 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026895ce-e3af-4de4-a583-629ad05a9c9c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:37.417864 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026895ce-e3af-4de4-a583-629ad05a9c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:37.418124 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026895ce-e3af-4de4-a583-629ad05a9c9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:37.433712 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026895ce-e3af-4de4-a583-629ad05a9c9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:37.435166 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026895ce-e3af-4de4-a583-629ad05a9c9c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:37.453452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-026895ce-e3af-4de4-a583-629ad05a9c9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:37.453452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:37.453452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:37.453452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:37.453452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:37.453452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:37.453452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:37.453452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:37.453452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:37.453452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:37.453452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:37.453452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:37.453452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:37.453452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:37.453452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:37.453452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:37.455414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:37.455414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:37.455414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:37.455414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:37.455414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:37.455414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:37.455414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:37.455414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:37.455414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:37.455414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:37.455414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:37.455414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:37.455414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:37.455414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:37.455414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:37.455414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:37.455414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:37.456564 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:37.456564 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:37.456564 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:37.456564 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 359/726] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:37.461080 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71f2ee-64fe-490b-bb58-397a064e3f51 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:37.553230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026895ce-e3af-4de4-a583-629ad05a9c9c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:37.577710 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71f2ee-64fe-490b-bb58-397a064e3f51 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:37.625410 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc21dbf-d56a-4007-91de-738f0738835b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:37.626379 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc21dbf-d56a-4007-91de-738f0738835b None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:37.626865 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc21dbf-d56a-4007-91de-738f0738835b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:37.627338 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc21dbf-d56a-4007-91de-738f0738835b None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:37.628216 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc21dbf-d56a-4007-91de-738f0738835b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:37.628744 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc21dbf-d56a-4007-91de-738f0738835b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:37.630316 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc21dbf-d56a-4007-91de-738f0738835b None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:37.649988 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 368/727] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:37.658192 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc21dbf-d56a-4007-91de-738f0738835b None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:37.742692 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:37.751375 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026895ce-e3af-4de4-a583-629ad05a9c9c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:37.753040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:37.809001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b532df62-d60c-40d8-aad6-d14faf9ccb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:37.809635 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532df62-d60c-40d8-aad6-d14faf9ccb5b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:37.809911 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532df62-d60c-40d8-aad6-d14faf9ccb5b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:37.810178 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532df62-d60c-40d8-aad6-d14faf9ccb5b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:37.832377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b532df62-d60c-40d8-aad6-d14faf9ccb5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:47:37.843903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-b532df62-d60c-40d8-aad6-d14faf9ccb5b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:37.843903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:37.843903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:37.843903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:37.843903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:37.843903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:37.843903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:37.843903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:37.843903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:37.843903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:37.843903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:37.843903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:37.843903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:47:37.843903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:47:37.843903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:47:37.843903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:47:37.844898 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:47:37.844898 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:47:37.844898 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:47:37.844898 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:47:37.844898 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:37.844898 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:37.844898 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:37.844898 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:37.844898 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:37.844898 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:37.844898 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:37.844898 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:37.844898 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:37.844898 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:37.844898 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:37.844898 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:37.845735 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:37.845735 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:37.845735 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:37.845735 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:37.845735 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:47:37.845735 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:47:37.845735 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:47:37.845735 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:47:37.845735 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:47:37.845735 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:47:37.845735 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:37.845735 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:37.848756 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 360/728] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Sep 9 20:47:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:47:37.850827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b532df62-d60c-40d8-aad6-d14faf9ccb5b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:37.873461 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc21dbf-d56a-4007-91de-738f0738835b None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:37.935434 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92d27f-09d4-4f2e-a322-8a6e5b41f803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:37.936472 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92d27f-09d4-4f2e-a322-8a6e5b41f803 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:37.936719 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92d27f-09d4-4f2e-a322-8a6e5b41f803 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:37.937019 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92d27f-09d4-4f2e-a322-8a6e5b41f803 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:37.953310 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b92d27f-09d4-4f2e-a322-8a6e5b41f803 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:37.954722 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b92d27f-09d4-4f2e-a322-8a6e5b41f803 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:37.985408 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-3b92d27f-09d4-4f2e-a322-8a6e5b41f803 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 20:47:37.985408 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:37.985408 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:37.985408 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:37.985408 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:37.985408 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:37.985408 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:37.985408 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:37.985408 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:37.985408 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:37.985408 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:37.985408 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:37.985408 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:47:37.985408 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:47:37.985408 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:47:37.985408 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:47:37.986813 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:37.986813 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:37.986813 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:47:37.986813 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:37.986813 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:47:37.986813 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:37.986813 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:37.986813 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:37.986813 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:37.986813 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:37.986813 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:37.986813 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:37.986813 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:37.986813 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:37.986813 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:37.986813 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:37.986813 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:37.986813 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:37.987739 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:37.987739 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:37.987739 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:47:37.987739 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:47:37.987739 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:47:37.987739 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:47:37.987739 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:47:37.987739 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:47:37.987739 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 20:47:37.987739 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:37.991796 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 369/729] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:47:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:47:37.993126 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92d27f-09d4-4f2e-a322-8a6e5b41f803 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:38.000500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b532df62-d60c-40d8-aad6-d14faf9ccb5b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:38.035893 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b8017-5cc4-4071-9140-2fd3f42e8a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:38.036937 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b8017-5cc4-4071-9140-2fd3f42e8a4b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:38.037444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b8017-5cc4-4071-9140-2fd3f42e8a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:38.037949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b8017-5cc4-4071-9140-2fd3f42e8a4b None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:38.038867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b8017-5cc4-4071-9140-2fd3f42e8a4b None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:38.039386 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b8017-5cc4-4071-9140-2fd3f42e8a4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:38.048103 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b8017-5cc4-4071-9140-2fd3f42e8a4b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:38.066058 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 361/730] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Sep 9 20:47:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:38.072260 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b8017-5cc4-4071-9140-2fd3f42e8a4b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:38.108126 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:38.156480 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:38.166282 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92d27f-09d4-4f2e-a322-8a6e5b41f803 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:38.205986 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3144b9-9959-491a-bd58-ae754529d985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:38.206506 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3144b9-9959-491a-bd58-ae754529d985 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:38.206781 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3144b9-9959-491a-bd58-ae754529d985 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:38.207049 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3144b9-9959-491a-bd58-ae754529d985 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:38.207771 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3144b9-9959-491a-bd58-ae754529d985 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:38.208097 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3144b9-9959-491a-bd58-ae754529d985 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:38.209531 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3144b9-9959-491a-bd58-ae754529d985 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:38.244093 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 370/731] 10.222.0.48 () {56 vars in 1135 bytes} [Fri Sep 9 20:47:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:47:38.249935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3144b9-9959-491a-bd58-ae754529d985 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:38.272600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b8017-5cc4-4071-9140-2fd3f42e8a4b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:38.343239 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e7180-0d0a-476b-9c62-ac79e18a3262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:38.343879 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e7180-0d0a-476b-9c62-ac79e18a3262 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:38.346329 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e7180-0d0a-476b-9c62-ac79e18a3262 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:38.346623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e7180-0d0a-476b-9c62-ac79e18a3262 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:38.373262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e7180-0d0a-476b-9c62-ac79e18a3262 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:38.374829 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e7180-0d0a-476b-9c62-ac79e18a3262 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:38.407593 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-507e7180-0d0a-476b-9c62-ac79e18a3262 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:38.407593 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:38.407593 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:38.407593 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:38.407593 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:38.407593 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:38.407593 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:38.407593 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:38.407593 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:38.407593 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:38.407593 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:38.407593 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:38.407593 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:38.407593 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:38.407593 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:38.407593 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:38.408616 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:38.408616 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:38.408616 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:38.408616 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:38.408616 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:38.408616 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:38.408616 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:38.408616 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:38.408616 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:38.408616 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:38.408616 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:38.408616 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:38.408616 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:38.408616 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:38.408616 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:38.408616 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:38.408616 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:38.409505 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:38.409505 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:38.409505 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:38.412368 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 362/732] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:38.418383 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e7180-0d0a-476b-9c62-ac79e18a3262 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:38.444639 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3144b9-9959-491a-bd58-ae754529d985 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:38.508866 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf461d94-c876-4c3f-9570-e3bd40d9e4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:38.509397 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf461d94-c876-4c3f-9570-e3bd40d9e4de None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:38.509673 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf461d94-c876-4c3f-9570-e3bd40d9e4de None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:38.509944 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf461d94-c876-4c3f-9570-e3bd40d9e4de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:38.533062 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:38.534780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:38.565532 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e7180-0d0a-476b-9c62-ac79e18a3262 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:38.583713 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf461d94-c876-4c3f-9570-e3bd40d9e4de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:38.585752 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf461d94-c876-4c3f-9570-e3bd40d9e4de None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:38.603580 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2126b-4b1f-4553-89bd-316bbd626aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:38.604433 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2126b-4b1f-4553-89bd-316bbd626aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:38.604858 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2126b-4b1f-4553-89bd-316bbd626aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:38.605273 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2126b-4b1f-4553-89bd-316bbd626aa5 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:38.606053 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2126b-4b1f-4553-89bd-316bbd626aa5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:38.606554 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2126b-4b1f-4553-89bd-316bbd626aa5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:38.608236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2126b-4b1f-4553-89bd-316bbd626aa5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:38.621845 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 363/733] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:47:38.623471 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d395553d-1290-48e2-9c62-d75aec0611dd None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:38.623905 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d395553d-1290-48e2-9c62-d75aec0611dd None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:38.624361 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d395553d-1290-48e2-9c62-d75aec0611dd None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:38.626319 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 364/734] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:38.627297 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d395553d-1290-48e2-9c62-d75aec0611dd None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:38.693987 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 371/735] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:47:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 20:47:38.695316 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c5be3-6df0-4ea8-bc50-142dfc4d0906 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:38.695668 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c5be3-6df0-4ea8-bc50-142dfc4d0906 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:38.696018 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c5be3-6df0-4ea8-bc50-142dfc4d0906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:38.715512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-b47c5be3-6df0-4ea8-bc50-142dfc4d0906 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:38.716879 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b47c5be3-6df0-4ea8-bc50-142dfc4d0906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:38.739510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d395553d-1290-48e2-9c62-d75aec0611dd None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:38.769383 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:38.772812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:38.779606 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3e6e2-42a4-487b-9b2e-41658874ec53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:38.780594 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3e6e2-42a4-487b-9b2e-41658874ec53 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:38.781040 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3e6e2-42a4-487b-9b2e-41658874ec53 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:38.781477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3e6e2-42a4-487b-9b2e-41658874ec53 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:38.789927 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3e6e2-42a4-487b-9b2e-41658874ec53 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:38.791619 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3e6e2-42a4-487b-9b2e-41658874ec53 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:38.793436 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 372/736] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:47:38.794506 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c5be3-6df0-4ea8-bc50-142dfc4d0906 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:38.805610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-23b3e6e2-42a4-487b-9b2e-41658874ec53 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 20:47:38.805610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:38.805610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:38.805610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:38.805610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:38.805610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:38.805610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:38.805610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:38.805610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:38.805610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:38.805610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:38.805610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:38.805610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:47:38.805610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:47:38.805610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:47:38.805610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:47:38.812577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:38.812577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:38.812577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:47:38.812577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:38.812577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:47:38.812577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:38.812577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:38.812577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:38.812577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:38.812577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:38.812577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:38.812577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:38.812577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:38.812577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:38.812577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:38.812577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:38.812577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:38.812577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:38.813535 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:38.813535 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:38.813535 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:47:38.813535 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:47:38.813535 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:47:38.813535 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:47:38.813535 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:47:38.813535 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:47:38.813535 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 20:47:38.813535 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:38.813535 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 365/737] 10.222.0.48 () {56 vars in 1117 bytes} [Fri Sep 9 20:47:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:38.815021 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3e6e2-42a4-487b-9b2e-41658874ec53 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:38.979177 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3e6e2-42a4-487b-9b2e-41658874ec53 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:38.984674 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c5be3-6df0-4ea8-bc50-142dfc4d0906 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:39.030448 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0f1fb-107c-413b-a092-402fda069ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:39.031299 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0f1fb-107c-413b-a092-402fda069ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:39.031727 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0f1fb-107c-413b-a092-402fda069ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:39.032177 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0f1fb-107c-413b-a092-402fda069ac2 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:39.033076 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca0f1fb-107c-413b-a092-402fda069ac2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:39.033542 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca0f1fb-107c-413b-a092-402fda069ac2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:39.035133 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca0f1fb-107c-413b-a092-402fda069ac2 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:39.055365 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 366/738] 10.222.0.48 () {56 vars in 1149 bytes} [Fri Sep 9 20:47:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:39.060337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0f1fb-107c-413b-a092-402fda069ac2 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:39.120473 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:39.121047 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484e885-a1d2-43da-9319-551667e8d1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:39.122002 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484e885-a1d2-43da-9319-551667e8d1f8 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:39.122380 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484e885-a1d2-43da-9319-551667e8d1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:39.122779 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484e885-a1d2-43da-9319-551667e8d1f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:39.143016 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3484e885-a1d2-43da-9319-551667e8d1f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:39.144796 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3484e885-a1d2-43da-9319-551667e8d1f8 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:39.169614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-3484e885-a1d2-43da-9319-551667e8d1f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:39.169614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:39.169614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:39.169614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:39.169614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:39.169614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:39.169614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:39.169614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:39.169614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:39.169614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:39.169614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:39.169614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:39.169614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:39.169614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:39.169614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:39.169614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:39.170703 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:39.170703 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:39.170703 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:39.170703 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:39.170703 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:39.170703 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:39.170703 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:39.170703 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:39.170703 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:39.170703 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:39.170703 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:39.170703 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:39.170703 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:39.170703 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:39.170703 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:39.170703 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:39.170703 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:39.171743 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:39.171743 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:39.171743 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:39.171743 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 373/739] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:39.172676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484e885-a1d2-43da-9319-551667e8d1f8 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:39.194981 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0f1fb-107c-413b-a092-402fda069ac2 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:39.219224 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:39.235392 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc2825-1159-4533-9996-a2370bf7c7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:39.236433 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc2825-1159-4533-9996-a2370bf7c7f4 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:39.236880 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc2825-1159-4533-9996-a2370bf7c7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:39.240243 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc2825-1159-4533-9996-a2370bf7c7f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:39.296440 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484e885-a1d2-43da-9319-551667e8d1f8 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:39.318014 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc2825-1159-4533-9996-a2370bf7c7f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:39.320420 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc2825-1159-4533-9996-a2370bf7c7f4 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:39.333887 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14673a-608f-4863-b493-27ef9e5d7963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:39.334527 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14673a-608f-4863-b493-27ef9e5d7963 None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:39.334812 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14673a-608f-4863-b493-27ef9e5d7963 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:39.335089 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14673a-608f-4863-b493-27ef9e5d7963 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:39.335739 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c14673a-608f-4863-b493-27ef9e5d7963 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:39.337330 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c14673a-608f-4863-b493-27ef9e5d7963 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:39.371746 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:39.404951 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 374/740] 10.222.0.48 () {60 vars in 1141 bytes} [Fri Sep 9 20:47:39 2022] POST /identity/v3/services => generated 251 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:47:39.405784 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14673a-608f-4863-b493-27ef9e5d7963 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:39.425335 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 367/741] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:47:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 20:47:39.426658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668cade-7b71-438f-91f0-61e849d269b8 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:39.426940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668cade-7b71-438f-91f0-61e849d269b8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:39.427215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668cade-7b71-438f-91f0-61e849d269b8 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:39.433426 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 368/742] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:39.433426 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8da68-a910-4b08-bc6e-275236f67fd1 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:39.433426 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8da68-a910-4b08-bc6e-275236f67fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:39.433426 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8da68-a910-4b08-bc6e-275236f67fd1 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:39.433426 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 369/743] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:39.433426 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8da68-a910-4b08-bc6e-275236f67fd1 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:39.541986 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:39.560037 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8da68-a910-4b08-bc6e-275236f67fd1 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:39.565485 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14673a-608f-4863-b493-27ef9e5d7963 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:39.600304 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b6e95c-dc22-4233-afac-fe504d661182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:39.601121 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6e95c-dc22-4233-afac-fe504d661182 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:39.601473 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6e95c-dc22-4233-afac-fe504d661182 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:39.601800 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6e95c-dc22-4233-afac-fe504d661182 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:39.608058 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7dbc7c-e24a-48b7-8fa3-c39262b3900b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:39.608729 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7dbc7c-e24a-48b7-8fa3-c39262b3900b None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:39.609210 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7dbc7c-e24a-48b7-8fa3-c39262b3900b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:39.610024 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7dbc7c-e24a-48b7-8fa3-c39262b3900b None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:39.610689 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7dbc7c-e24a-48b7-8fa3-c39262b3900b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:39.610999 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7dbc7c-e24a-48b7-8fa3-c39262b3900b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:39.612428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7dbc7c-e24a-48b7-8fa3-c39262b3900b None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:39.613152 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b6e95c-dc22-4233-afac-fe504d661182 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:39.614550 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b6e95c-dc22-4233-afac-fe504d661182 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:39.618048 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:39.627105 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 375/744] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:39.627105 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b19a8-8310-43c5-9a2f-16c6f999147a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:39.627363 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b19a8-8310-43c5-9a2f-16c6f999147a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:39.627487 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b19a8-8310-43c5-9a2f-16c6f999147a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:39.634420 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-e2b6e95c-dc22-4233-afac-fe504d661182 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:39.634420 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:39.634420 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:39.634420 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:39.634420 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:39.634420 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:39.634420 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:39.634420 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:39.634420 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:39.634420 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:39.634420 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:39.634420 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:39.634420 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:39.634420 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:39.634420 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:39.634420 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:39.635573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:39.635573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:39.635573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:39.635573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:39.635573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:39.635573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:39.635573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:39.635573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:39.635573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:39.635573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:39.635573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:39.635573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:39.635573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:39.635573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:39.635573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:39.635573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:39.635573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:39.636757 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:39.636757 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:39.636757 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:39.636757 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 370/745] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:39.637571 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d74fd-beec-4d7f-b870-6a278fd40ad8 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:39.637851 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d74fd-beec-4d7f-b870-6a278fd40ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:39.638124 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d74fd-beec-4d7f-b870-6a278fd40ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:39.649014 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-331b19a8-8310-43c5-9a2f-16c6f999147a None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:39.650361 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-331b19a8-8310-43c5-9a2f-16c6f999147a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:39.656768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-db0d74fd-beec-4d7f-b870-6a278fd40ad8 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:39.657918 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-db0d74fd-beec-4d7f-b870-6a278fd40ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:39.721842 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 376/746] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:39 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:47:39.723044 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b19a8-8310-43c5-9a2f-16c6f999147a None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:39.739171 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 371/747] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:39 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 20:47:39.739171 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d74fd-beec-4d7f-b870-6a278fd40ad8 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:39.865288 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:39.901510 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b19a8-8310-43c5-9a2f-16c6f999147a None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:39.922425 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d74fd-beec-4d7f-b870-6a278fd40ad8 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:39.954730 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:39.960505 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ced18-6302-44a3-beea-92970d9691f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:39.961516 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ced18-6302-44a3-beea-92970d9691f2 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:39.961895 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ced18-6302-44a3-beea-92970d9691f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:39.962264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ced18-6302-44a3-beea-92970d9691f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:39.962978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ced18-6302-44a3-beea-92970d9691f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:39.963338 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ced18-6302-44a3-beea-92970d9691f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:39.964797 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ced18-6302-44a3-beea-92970d9691f2 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:39.977789 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 372/748] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:39.982977 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ced18-6302-44a3-beea-92970d9691f2 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:39.993322 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc43b9-d8f4-4e3e-80ea-d662dbf86eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:39.993974 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc43b9-d8f4-4e3e-80ea-d662dbf86eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:39.994248 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc43b9-d8f4-4e3e-80ea-d662dbf86eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:39.994514 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc43b9-d8f4-4e3e-80ea-d662dbf86eb5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:40.017147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fc43b9-d8f4-4e3e-80ea-d662dbf86eb5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:40.018591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fc43b9-d8f4-4e3e-80ea-d662dbf86eb5 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:40.041428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-f2fc43b9-d8f4-4e3e-80ea-d662dbf86eb5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:40.041428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:40.041428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:40.041428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:40.041428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:40.041428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:40.041428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:40.041428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:40.041428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:40.041428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:40.041428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:40.041428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:40.041428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:47:40.041428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:47:40.041428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:47:40.042363 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:47:40.042363 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:40.042363 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:40.042363 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:40.042363 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:40.042363 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:40.042363 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:40.042363 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:40.042363 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:40.042363 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:40.042363 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:40.042363 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:40.042363 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:40.042363 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:40.042363 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:40.042363 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:40.042363 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:40.043249 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:47:40.043249 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:47:40.043249 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:47:40.043249 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:47:40.043249 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:47:40.043249 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:47:40.043249 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:40.043249 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:40.043249 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 377/749] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:47:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:40.043249 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc43b9-d8f4-4e3e-80ea-d662dbf86eb5 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:40.094675 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ced18-6302-44a3-beea-92970d9691f2 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:40.132612 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e31c1f-a4b5-4aab-91f6-e39912f5a22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:40.133268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e31c1f-a4b5-4aab-91f6-e39912f5a22e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:40.133554 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e31c1f-a4b5-4aab-91f6-e39912f5a22e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:40.133824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e31c1f-a4b5-4aab-91f6-e39912f5a22e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:40.140963 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e31c1f-a4b5-4aab-91f6-e39912f5a22e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:40.142574 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e31c1f-a4b5-4aab-91f6-e39912f5a22e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:40.154722 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc43b9-d8f4-4e3e-80ea-d662dbf86eb5 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:40.155309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-45e31c1f-a4b5-4aab-91f6-e39912f5a22e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:40.155309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:40.155309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:40.155309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:40.155309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:40.155309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:40.155309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:40.155309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:40.155309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:40.155309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:40.155309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:40.155309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:40.155309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:47:40.155309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:47:40.155309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:47:40.156254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:47:40.156254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:40.156254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:40.156254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:40.156254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:40.156254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:40.156254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:40.156254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:40.156254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:40.156254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:40.156254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:40.156254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:40.156254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:40.156254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:40.156254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:40.156254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:40.156254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:40.157140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:47:40.157140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:47:40.157140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:47:40.157140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:47:40.157140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:47:40.157140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:47:40.157140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:40.157140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:40.157140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 373/750] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:47:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:40.157620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e31c1f-a4b5-4aab-91f6-e39912f5a22e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:40.198530 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8fa40-b669-4d80-b325-8d12b5a00260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:40.199140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8fa40-b669-4d80-b325-8d12b5a00260 None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:40.199418 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8fa40-b669-4d80-b325-8d12b5a00260 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:40.199854 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8fa40-b669-4d80-b325-8d12b5a00260 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:40.200545 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8fa40-b669-4d80-b325-8d12b5a00260 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:40.202063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8fa40-b669-4d80-b325-8d12b5a00260 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:40.207187 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:40.259312 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e31c1f-a4b5-4aab-91f6-e39912f5a22e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:40.293477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36170dbf-ce62-4359-97f6-84adcb31bfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:40.294123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36170dbf-ce62-4359-97f6-84adcb31bfaf None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:40.294441 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36170dbf-ce62-4359-97f6-84adcb31bfaf None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:40.294793 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36170dbf-ce62-4359-97f6-84adcb31bfaf None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:40.295458 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36170dbf-ce62-4359-97f6-84adcb31bfaf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:40.295758 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36170dbf-ce62-4359-97f6-84adcb31bfaf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:40.297277 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36170dbf-ce62-4359-97f6-84adcb31bfaf None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:40.308490 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:40.315777 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 374/751] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:47:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:40.315777 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36170dbf-ce62-4359-97f6-84adcb31bfaf None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:40.321828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 378/752] 10.222.0.48 () {60 vars in 1141 bytes} [Fri Sep 9 20:47:40 2022] POST /identity/v3/services => generated 240 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:47:40.321828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8fa40-b669-4d80-b325-8d12b5a00260 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:40.437247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36170dbf-ce62-4359-97f6-84adcb31bfaf None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:40.446419 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:40.470851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8fa40-b669-4d80-b325-8d12b5a00260 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:40.484167 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c233bc-a638-43f8-a5cb-a38b4d7450c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:40.485487 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c233bc-a638-43f8-a5cb-a38b4d7450c9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:40.485999 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c233bc-a638-43f8-a5cb-a38b4d7450c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:40.486683 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c233bc-a638-43f8-a5cb-a38b4d7450c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:40.503404 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c233bc-a638-43f8-a5cb-a38b4d7450c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:47:40.511078 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-29c233bc-a638-43f8-a5cb-a38b4d7450c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:47:40.511078 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:40.511078 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:40.511078 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:40.511078 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:40.511078 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:40.511078 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:40.511078 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:40.511078 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:40.511078 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:40.511078 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:40.511078 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:40.511078 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:47:40.511078 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:47:40.511078 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:47:40.511078 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:47:40.512020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:47:40.512020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:47:40.512020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:47:40.512020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:47:40.512020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:40.512020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:40.512020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:40.512020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:40.512020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:40.512020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:40.512020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:40.512020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:40.512020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:40.512020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:40.512020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:40.512020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:40.512912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:40.512912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:40.512912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:40.512912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:40.512912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:47:40.512912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:47:40.512912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:47:40.512912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:47:40.512912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:47:40.512912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:47:40.512912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:47:40.512912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:40.513550 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255060e-515d-42c4-9e42-f4efd85781a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:40.513550 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255060e-515d-42c4-9e42-f4efd85781a7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:40.513550 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255060e-515d-42c4-9e42-f4efd85781a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:40.513550 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255060e-515d-42c4-9e42-f4efd85781a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:40.513813 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f255060e-515d-42c4-9e42-f4efd85781a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:40.513813 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f255060e-515d-42c4-9e42-f4efd85781a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:40.514873 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f255060e-515d-42c4-9e42-f4efd85781a7 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:40.515937 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 375/753] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:47:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:40.516969 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c233bc-a638-43f8-a5cb-a38b4d7450c9 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:40.531124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 379/754] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:47:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:40.531962 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255060e-515d-42c4-9e42-f4efd85781a7 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:40.603647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:40.652746 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255060e-515d-42c4-9e42-f4efd85781a7 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:40.665563 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c233bc-a638-43f8-a5cb-a38b4d7450c9 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:40.691145 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708b513-7702-485a-8239-94858bdaf32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:40.691683 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708b513-7702-485a-8239-94858bdaf32f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:40.691960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708b513-7702-485a-8239-94858bdaf32f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:40.692264 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708b513-7702-485a-8239-94858bdaf32f None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:40.692935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b708b513-7702-485a-8239-94858bdaf32f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:40.693236 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b708b513-7702-485a-8239-94858bdaf32f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:40.695366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b708b513-7702-485a-8239-94858bdaf32f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:40.704351 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 380/755] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:47:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:40.705457 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708b513-7702-485a-8239-94858bdaf32f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:40.706894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab14c89-f1a4-44bc-b312-5612b42bcb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:40.707506 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab14c89-f1a4-44bc-b312-5612b42bcb46 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:40.707784 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab14c89-f1a4-44bc-b312-5612b42bcb46 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:40.708146 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab14c89-f1a4-44bc-b312-5612b42bcb46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:40.710696 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab14c89-f1a4-44bc-b312-5612b42bcb46 None admin] RBAC: Adding query filter params (role.id=fcbf519fbc84458f913671a5a1d0914c, scope.project.id=91319aa782334fb9930770029b74e84a, user.id=7d28fa5d49d44120bd5eaec2d4e5fdbd) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:40.711022 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab14c89-f1a4-44bc-b312-5612b42bcb46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:40.712541 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab14c89-f1a4-44bc-b312-5612b42bcb46 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:40.720269 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 376/756] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:47:40 2022] GET /identity/v3/role_assignments?user.id=7d28fa5d49d44120bd5eaec2d4e5fdbd&scope.project.id=91319aa782334fb9930770029b74e84a&role.id=fcbf519fbc84458f913671a5a1d0914c => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:40.721192 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab14c89-f1a4-44bc-b312-5612b42bcb46 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:40.776534 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:40.839558 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708b513-7702-485a-8239-94858bdaf32f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:40.860327 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:40.880887 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ec7a6-788f-40f9-a55a-ea862f0c00a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:40.882073 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ec7a6-788f-40f9-a55a-ea862f0c00a7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:40.882353 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ec7a6-788f-40f9-a55a-ea862f0c00a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:40.882629 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ec7a6-788f-40f9-a55a-ea862f0c00a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:40.883371 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78ec7a6-788f-40f9-a55a-ea862f0c00a7 None admin] RBAC: Adding query filter params (role.id=1e000597af164d54a40df2a39c729820, scope.project.id=91319aa782334fb9930770029b74e84a, user.id=8c21ea7d3cbb494886e60a4bd4f39145) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:40.883683 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78ec7a6-788f-40f9-a55a-ea862f0c00a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:40.885489 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78ec7a6-788f-40f9-a55a-ea862f0c00a7 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:40.896683 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 381/757] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:47:40 2022] GET /identity/v3/role_assignments?user.id=8c21ea7d3cbb494886e60a4bd4f39145&scope.project.id=91319aa782334fb9930770029b74e84a&role.id=1e000597af164d54a40df2a39c729820 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:40.934654 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab14c89-f1a4-44bc-b312-5612b42bcb46 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:40.986106 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df875f4-6a1c-490f-870f-87c9d93cf1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:40.986106 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df875f4-6a1c-490f-870f-87c9d93cf1ee None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:40.986106 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df875f4-6a1c-490f-870f-87c9d93cf1ee None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:40.986106 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df875f4-6a1c-490f-870f-87c9d93cf1ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:40.997948 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df875f4-6a1c-490f-870f-87c9d93cf1ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:40.999451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df875f4-6a1c-490f-870f-87c9d93cf1ee None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:41.015997 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-4df875f4-6a1c-490f-870f-87c9d93cf1ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:41.015997 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:41.015997 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:41.015997 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:41.015997 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:41.015997 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:41.015997 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:41.015997 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:41.015997 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:41.015997 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:41.015997 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:41.015997 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:41.015997 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:41.015997 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:41.015997 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:41.015997 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:41.016993 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:41.016993 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:41.016993 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:41.016993 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:41.016993 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:41.016993 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:41.016993 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:41.016993 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:41.016993 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:41.016993 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:41.016993 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:41.016993 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:41.016993 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:41.016993 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:41.016993 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:41.016993 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:41.016993 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:41.017874 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:41.017874 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:41.017874 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:41.017874 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 377/758] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:41.022934 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ec7a6-788f-40f9-a55a-ea862f0c00a7 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:41.132125 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ec7a6-788f-40f9-a55a-ea862f0c00a7 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:41.167255 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dffc273-3032-412d-b4e8-ccebc4e602bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:41.167886 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffc273-3032-412d-b4e8-ccebc4e602bb None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:41.168172 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffc273-3032-412d-b4e8-ccebc4e602bb None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:41.168450 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffc273-3032-412d-b4e8-ccebc4e602bb None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:41.169104 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dffc273-3032-412d-b4e8-ccebc4e602bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:41.169421 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dffc273-3032-412d-b4e8-ccebc4e602bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:41.170791 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dffc273-3032-412d-b4e8-ccebc4e602bb None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:41.184093 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 382/759] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:47:41.193265 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df875f4-6a1c-490f-870f-87c9d93cf1ee None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:41.309323 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:41.329481 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df875f4-6a1c-490f-870f-87c9d93cf1ee None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:41.365448 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd890e-74f1-4a32-a6f9-f94652aba6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:41.367021 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd890e-74f1-4a32-a6f9-f94652aba6dd None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:41.367504 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd890e-74f1-4a32-a6f9-f94652aba6dd None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:41.367976 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd890e-74f1-4a32-a6f9-f94652aba6dd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:41.376191 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd890e-74f1-4a32-a6f9-f94652aba6dd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:41.377883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd890e-74f1-4a32-a6f9-f94652aba6dd None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:41.389857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-bfbd890e-74f1-4a32-a6f9-f94652aba6dd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:47:41.389857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:41.389857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:41.389857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:41.389857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:41.389857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:41.389857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:41.389857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:41.389857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:41.389857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:41.389857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:41.389857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:41.389857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:47:41.389857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:47:41.389857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:47:41.389857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:47:41.390830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:41.390830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:41.390830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:47:41.390830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:41.390830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:47:41.390830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:41.390830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:41.390830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:41.390830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:41.390830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:41.390830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:41.390830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:41.390830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:41.390830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:41.390830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:41.390830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:41.390830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:41.390830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:41.391776 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:41.391776 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:41.391776 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:47:41.391776 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:47:41.391776 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:47:41.391776 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:47:41.391776 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:47:41.391776 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:47:41.391776 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:47:41.391776 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:41.394352 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 378/760] 10.222.0.48 () {56 vars in 1093 bytes} [Fri Sep 9 20:47:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:41.399678 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dffc273-3032-412d-b4e8-ccebc4e602bb None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:41.481941 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:41.540289 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dffc273-3032-412d-b4e8-ccebc4e602bb None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:41.575261 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b697895-4cb9-4148-9c04-1bd03c56f1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:41.575800 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b697895-4cb9-4148-9c04-1bd03c56f1d3 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:41.576096 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b697895-4cb9-4148-9c04-1bd03c56f1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:41.578025 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b697895-4cb9-4148-9c04-1bd03c56f1d3 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:41.578778 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b697895-4cb9-4148-9c04-1bd03c56f1d3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:41.579091 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b697895-4cb9-4148-9c04-1bd03c56f1d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:41.580617 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b697895-4cb9-4148-9c04-1bd03c56f1d3 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:41.602296 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 383/761] 10.222.0.48 () {56 vars in 1133 bytes} [Fri Sep 9 20:47:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:41.607535 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd890e-74f1-4a32-a6f9-f94652aba6dd None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:41.779447 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd890e-74f1-4a32-a6f9-f94652aba6dd None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:41.841445 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2d641-ca7e-47c2-b2ca-124694732e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:41.842452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2d641-ca7e-47c2-b2ca-124694732e4d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:41.842942 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2d641-ca7e-47c2-b2ca-124694732e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:41.843433 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2d641-ca7e-47c2-b2ca-124694732e4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:41.859765 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad2d641-ca7e-47c2-b2ca-124694732e4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:41.861599 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad2d641-ca7e-47c2-b2ca-124694732e4d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:41.863630 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:41.877222 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-7ad2d641-ca7e-47c2-b2ca-124694732e4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:41.877222 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:41.877222 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:41.877222 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:41.877222 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:41.877222 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:41.877222 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:41.877222 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:41.877222 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:41.877222 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:41.877222 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:41.877222 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:41.877222 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:41.877222 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:41.877222 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:41.877222 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:41.882768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:41.882768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:41.882768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:41.882768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:41.882768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:41.882768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:41.882768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:41.882768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:41.882768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:41.882768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:41.882768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:41.882768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:41.882768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:41.882768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:41.882768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:41.882768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:41.882768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:41.883661 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:41.883661 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:41.883661 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:41.883661 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 379/762] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:41.886051 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde5e48-a9cb-4410-9a31-3bb50d44c9b9 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:41.886342 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde5e48-a9cb-4410-9a31-3bb50d44c9b9 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:41.886701 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde5e48-a9cb-4410-9a31-3bb50d44c9b9 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:41.887833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2d641-ca7e-47c2-b2ca-124694732e4d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:41.894164 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 384/763] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:41.906308 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a7fe5-a622-44c0-b844-44ca6631c964 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:41.906594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a7fe5-a622-44c0-b844-44ca6631c964 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:41.906859 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a7fe5-a622-44c0-b844-44ca6631c964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:41.941349 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-608a7fe5-a622-44c0-b844-44ca6631c964 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:41.942551 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-608a7fe5-a622-44c0-b844-44ca6631c964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:42.045348 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 385/764] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:47:42.083275 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2d641-ca7e-47c2-b2ca-124694732e4d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:42.118919 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63ce64-3d2e-4269-8def-435d9ae25497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:42.119456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63ce64-3d2e-4269-8def-435d9ae25497 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:42.119785 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63ce64-3d2e-4269-8def-435d9ae25497 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:42.120063 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63ce64-3d2e-4269-8def-435d9ae25497 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:42.120732 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc63ce64-3d2e-4269-8def-435d9ae25497 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:42.121035 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc63ce64-3d2e-4269-8def-435d9ae25497 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:42.122513 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc63ce64-3d2e-4269-8def-435d9ae25497 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:42.135719 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 380/765] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:42.145166 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a7fe5-a622-44c0-b844-44ca6631c964 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:42.184920 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:42.294683 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a7fe5-a622-44c0-b844-44ca6631c964 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:42.354236 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbae3a7-177f-4d24-81cc-3a70e1aba0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:42.354958 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbae3a7-177f-4d24-81cc-3a70e1aba0ab None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:42.355235 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbae3a7-177f-4d24-81cc-3a70e1aba0ab None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:42.355504 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbae3a7-177f-4d24-81cc-3a70e1aba0ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:42.364707 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbae3a7-177f-4d24-81cc-3a70e1aba0ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:42.366707 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbae3a7-177f-4d24-81cc-3a70e1aba0ab None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:42.384159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-8fbae3a7-177f-4d24-81cc-3a70e1aba0ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:42.384159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:42.384159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:42.384159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:42.384159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:42.384159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:42.384159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:42.384159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:42.384159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:42.384159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:42.384159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:42.384159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:42.384159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:47:42.384159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:47:42.384159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:47:42.385034 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:47:42.385034 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:42.385034 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:42.385034 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:42.385034 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:42.385034 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:42.385034 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:42.385034 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:42.385034 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:42.385034 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:42.385034 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:42.385034 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:42.385034 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:42.385034 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:42.385034 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:42.385034 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:42.385034 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:42.385940 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:47:42.385940 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:47:42.385940 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:47:42.385940 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:47:42.385940 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:47:42.385940 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:47:42.385940 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:42.385940 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:42.385940 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 386/766] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:47:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:42.393350 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63ce64-3d2e-4269-8def-435d9ae25497 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:42.415905 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbae3a7-177f-4d24-81cc-3a70e1aba0ab None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:42.501870 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:42.572482 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63ce64-3d2e-4269-8def-435d9ae25497 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:42.573653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:42.583607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbae3a7-177f-4d24-81cc-3a70e1aba0ab None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:42.633071 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d8214-5e7a-4d39-bbf7-778ae4ff37f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:42.633700 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d8214-5e7a-4d39-bbf7-778ae4ff37f4 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:42.633985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d8214-5e7a-4d39-bbf7-778ae4ff37f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:42.634337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d8214-5e7a-4d39-bbf7-778ae4ff37f4 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:42.634954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d8214-5e7a-4d39-bbf7-778ae4ff37f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:42.635261 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d8214-5e7a-4d39-bbf7-778ae4ff37f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:42.637572 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d8214-5e7a-4d39-bbf7-778ae4ff37f4 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:42.654819 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 381/767] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:47:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:42.655082 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe55d4-d9f8-4031-86c2-8b6ae8e31575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:42.655676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe55d4-d9f8-4031-86c2-8b6ae8e31575 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:42.655951 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe55d4-d9f8-4031-86c2-8b6ae8e31575 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:42.656188 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d8214-5e7a-4d39-bbf7-778ae4ff37f4 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:42.656289 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe55d4-d9f8-4031-86c2-8b6ae8e31575 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:42.656823 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fe55d4-d9f8-4031-86c2-8b6ae8e31575 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:42.658177 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fe55d4-d9f8-4031-86c2-8b6ae8e31575 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:42.683412 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-81fe55d4-d9f8-4031-86c2-8b6ae8e31575 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 20:47:42.683412 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:42.683412 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:47:42.683412 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:47:42.683412 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:47:42.683412 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:47:42.683412 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:47:42.683412 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:47:42.683412 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 20:47:42.683412 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:42.683412 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:47:42.683412 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:42.683412 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:42.683412 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:42.683412 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:42.683412 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:42.683412 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:42.684502 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:42.684502 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:42.684502 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:42.684502 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:42.684502 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:42.684502 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:42.684502 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:47:42.684502 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:47:42.684502 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:47:42.684502 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:47:42.684502 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:42.684502 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:42.684502 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:42.684502 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:42.684502 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:42.684502 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:42.685324 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:42.685324 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:42.685324 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:42.685324 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:42.685324 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:42.685324 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:42.685324 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:42.685324 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:42.685324 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:42.685324 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:42.685324 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:47:42.685324 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:47:42.685324 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 20:47:42.685324 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:42.685324 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 387/768] 10.222.0.48 () {56 vars in 1129 bytes} [Fri Sep 9 20:47:42 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:47:42.692420 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe55d4-d9f8-4031-86c2-8b6ae8e31575 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:42.799654 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d8214-5e7a-4d39-bbf7-778ae4ff37f4 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:42.821482 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:42.862246 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3388b84-c92d-47a9-816b-00b2e5142b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:42.862911 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3388b84-c92d-47a9-816b-00b2e5142b2a None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:42.863191 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3388b84-c92d-47a9-816b-00b2e5142b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:42.863463 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3388b84-c92d-47a9-816b-00b2e5142b2a None admin] PATH_INFO: `/v3/projects/91319aa782334fb9930770029b74e84a/users/613b845a9b4d432daf8d478fc3970708/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:42.866519 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe55d4-d9f8-4031-86c2-8b6ae8e31575 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:42.869783 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3388b84-c92d-47a9-816b-00b2e5142b2a None admin] RBAC: Authorizing `identity:create_grant(project_id=91319aa782334fb9930770029b74e84a, user_id=613b845a9b4d432daf8d478fc3970708, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:42.875285 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3388b84-c92d-47a9-816b-00b2e5142b2a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:42.911738 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d401ac-fdf7-4437-abce-340d4920769d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:42.912407 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d401ac-fdf7-4437-abce-340d4920769d None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:42.913228 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d401ac-fdf7-4437-abce-340d4920769d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:42.913504 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d401ac-fdf7-4437-abce-340d4920769d None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:42.914161 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d401ac-fdf7-4437-abce-340d4920769d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:42.914464 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d401ac-fdf7-4437-abce-340d4920769d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:42.915861 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d401ac-fdf7-4437-abce-340d4920769d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:42.921517 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 388/769] 10.222.0.48 () {56 vars in 1124 bytes} [Fri Sep 9 20:47:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:42.930157 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d401ac-fdf7-4437-abce-340d4920769d None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:42.948317 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:42.974544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 382/770] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Sep 9 20:47:42 2022] PUT /identity/v3/projects/91319aa782334fb9930770029b74e84a/users/613b845a9b4d432daf8d478fc3970708/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:47:43.014003 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9ee0d-29cc-4440-8dd4-14d7e1d916b6 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:43.014003 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9ee0d-29cc-4440-8dd4-14d7e1d916b6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:43.014337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9ee0d-29cc-4440-8dd4-14d7e1d916b6 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:43.016936 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 383/771] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:47:43.021130 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783fbc8-5d5a-4867-a433-e493cd72f50f None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:43.021633 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783fbc8-5d5a-4867-a433-e493cd72f50f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:43.022596 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783fbc8-5d5a-4867-a433-e493cd72f50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:43.052459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d783fbc8-5d5a-4867-a433-e493cd72f50f None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:43.053954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d783fbc8-5d5a-4867-a433-e493cd72f50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:43.074615 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d401ac-fdf7-4437-abce-340d4920769d None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:43.155154 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0668c-bfda-49ef-a589-1a56b0a2a9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:43.156633 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0668c-bfda-49ef-a589-1a56b0a2a9c3 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:43.156915 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0668c-bfda-49ef-a589-1a56b0a2a9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:43.157181 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0668c-bfda-49ef-a589-1a56b0a2a9c3 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:43.157839 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae0668c-bfda-49ef-a589-1a56b0a2a9c3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:43.158144 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae0668c-bfda-49ef-a589-1a56b0a2a9c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:43.159541 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae0668c-bfda-49ef-a589-1a56b0a2a9c3 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:43.173036 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 389/772] 10.222.0.48 () {56 vars in 1124 bytes} [Fri Sep 9 20:47:42 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:43.179015 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0668c-bfda-49ef-a589-1a56b0a2a9c3 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:43.203777 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 384/773] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:47:43.268237 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:43.287160 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0668c-bfda-49ef-a589-1a56b0a2a9c3 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:43.326359 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89836ff0-ec97-43b4-9d83-98bda2f81cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:43.326999 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89836ff0-ec97-43b4-9d83-98bda2f81cf9 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:43.327277 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89836ff0-ec97-43b4-9d83-98bda2f81cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:43.327548 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89836ff0-ec97-43b4-9d83-98bda2f81cf9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:43.328297 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89836ff0-ec97-43b4-9d83-98bda2f81cf9 None admin] RBAC: Adding query filter params (interface=public, service_id=28962bddae994b1688e7c6ab542022aa, region_id=RegionOne) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:43.328603 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89836ff0-ec97-43b4-9d83-98bda2f81cf9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:43.330088 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89836ff0-ec97-43b4-9d83-98bda2f81cf9 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:43.339687 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 390/774] 10.222.0.48 () {56 vars in 1251 bytes} [Fri Sep 9 20:47:43 2022] GET /identity/v3/endpoints?service_id=28962bddae994b1688e7c6ab542022aa&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:43.345377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783fbc8-5d5a-4867-a433-e493cd72f50f None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:43.496855 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89836ff0-ec97-43b4-9d83-98bda2f81cf9 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:43.519690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783fbc8-5d5a-4867-a433-e493cd72f50f None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:43.528459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:43.555980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb712e-c851-47d7-bf12-8d9fe63dbead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:43.556620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb712e-c851-47d7-bf12-8d9fe63dbead None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:43.556906 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb712e-c851-47d7-bf12-8d9fe63dbead None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:43.557180 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb712e-c851-47d7-bf12-8d9fe63dbead None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:43.557790 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb712e-c851-47d7-bf12-8d9fe63dbead None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:43.559182 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb712e-c851-47d7-bf12-8d9fe63dbead None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:43.564883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 385/775] 10.222.0.48 () {56 vars in 1087 bytes} [Fri Sep 9 20:47:43 2022] GET /identity/v3/services => generated 772 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:43.565898 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ec2e3-4027-432a-92c4-4af86a180da7 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:43.566184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ec2e3-4027-432a-92c4-4af86a180da7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:43.566459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ec2e3-4027-432a-92c4-4af86a180da7 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:43.572581 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 386/776] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:43.574175 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f4223-536d-4cc1-8a78-27d353e0742c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:43.574456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f4223-536d-4cc1-8a78-27d353e0742c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:43.574730 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f4223-536d-4cc1-8a78-27d353e0742c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:43.595201 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-087f4223-536d-4cc1-8a78-27d353e0742c None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:43.600120 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-087f4223-536d-4cc1-8a78-27d353e0742c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:43.661884 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89836ff0-ec97-43b4-9d83-98bda2f81cf9 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:43.695731 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 387/777] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:47:43.697528 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da933ed3-d842-495f-9f42-d4e0798d6181 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:43.698134 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da933ed3-d842-495f-9f42-d4e0798d6181 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:43.699514 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da933ed3-d842-495f-9f42-d4e0798d6181 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:43.701428 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 388/778] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:43.712099 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f6097-b6a1-4c1a-9463-83688bf72866 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:43.712099 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f6097-b6a1-4c1a-9463-83688bf72866 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:43.712099 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f6097-b6a1-4c1a-9463-83688bf72866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:43.721957 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c60e4b-9f31-4190-9459-44de0b4c038f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:43.722621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c60e4b-9f31-4190-9459-44de0b4c038f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:43.722900 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c60e4b-9f31-4190-9459-44de0b4c038f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:43.723172 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c60e4b-9f31-4190-9459-44de0b4c038f None admin] PATH_INFO: `/v3/services/network` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:43.723771 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c60e4b-9f31-4190-9459-44de0b4c038f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:43.725231 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c60e4b-9f31-4190-9459-44de0b4c038f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:43.735376 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-b7c60e4b-9f31-4190-9459-44de0b4c038f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 20:47:43.735376 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:43.735376 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:47:43.735376 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:47:43.735376 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:47:43.735376 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:47:43.735376 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:47:43.735376 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:47:43.735376 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 20:47:43.735376 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:43.735376 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:47:43.735376 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:43.735376 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:43.735376 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:43.735376 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:43.735376 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:43.735376 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:43.736437 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:43.736437 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:43.736437 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:43.736437 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:43.736437 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:43.736437 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:43.736437 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:47:43.736437 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:47:43.736437 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:47:43.736437 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:47:43.736437 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:43.736437 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:43.736437 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:43.736437 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:43.736437 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:43.736437 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:43.737275 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:43.737275 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:43.737275 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:43.737275 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:43.737275 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:43.737275 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:43.737275 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:43.737275 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:43.737275 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:43.737275 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:43.737275 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:47:43.737275 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:47:43.737275 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 20:47:43.737275 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:43.737275 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 391/779] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:47:43 2022] GET /identity/v3/services/network => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:43.743533 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9a0f6097-b6a1-4c1a-9463-83688bf72866 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:43.744794 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9a0f6097-b6a1-4c1a-9463-83688bf72866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:43.751557 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c60e4b-9f31-4190-9459-44de0b4c038f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:43.768797 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:43.872705 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 389/780] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:47:43.874883 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c60e4b-9f31-4190-9459-44de0b4c038f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:43.910629 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d47f2-e74c-477f-a490-1cda2676098e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:43.911241 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d47f2-e74c-477f-a490-1cda2676098e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:43.911520 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d47f2-e74c-477f-a490-1cda2676098e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:43.911790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d47f2-e74c-477f-a490-1cda2676098e None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:43.912480 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388d47f2-e74c-477f-a490-1cda2676098e None admin] RBAC: Adding query filter params (name=network) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:43.912787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388d47f2-e74c-477f-a490-1cda2676098e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:43.914215 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388d47f2-e74c-477f-a490-1cda2676098e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:43.919048 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 392/781] 10.222.0.48 () {56 vars in 1112 bytes} [Fri Sep 9 20:47:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:47:43.924331 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f6097-b6a1-4c1a-9463-83688bf72866 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:43.955825 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:44.002387 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d47f2-e74c-477f-a490-1cda2676098e None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:44.086789 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f6097-b6a1-4c1a-9463-83688bf72866 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:44.092500 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:44.112803 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d47f2-e74c-477f-a490-1cda2676098e None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:44.151226 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b14b579-5a40-458a-9eb8-20c880ca2123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:44.151868 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14b579-5a40-458a-9eb8-20c880ca2123 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:44.152175 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14b579-5a40-458a-9eb8-20c880ca2123 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:44.152450 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14b579-5a40-458a-9eb8-20c880ca2123 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:44.162990 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b14b579-5a40-458a-9eb8-20c880ca2123 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:47:44.167983 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-3b14b579-5a40-458a-9eb8-20c880ca2123 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:47:44.167983 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:44.167983 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:44.167983 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:44.167983 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:44.167983 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:44.167983 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:44.167983 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:44.167983 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:44.167983 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:44.167983 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:44.167983 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:44.167983 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:47:44.167983 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:47:44.167983 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:47:44.167983 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:47:44.168931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:47:44.168931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:47:44.168931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:47:44.168931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:47:44.168931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:44.168931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:44.168931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:44.168931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:44.168931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:44.168931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:44.168931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:44.168931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:44.168931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:44.168931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:44.168931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:44.168931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:44.169790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:44.169790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:44.169790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:44.169790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:44.169790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:47:44.169790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:47:44.169790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:47:44.169790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:47:44.169790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:47:44.169790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:47:44.169790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:47:44.169790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:44.169790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 393/782] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:47:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:47:44.175309 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b14b579-5a40-458a-9eb8-20c880ca2123 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:44.186190 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b2dbe-5756-4458-bf3c-16b909c3da73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:44.186190 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b2dbe-5756-4458-bf3c-16b909c3da73 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:44.186190 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b2dbe-5756-4458-bf3c-16b909c3da73 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:44.186190 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b2dbe-5756-4458-bf3c-16b909c3da73 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:44.186914 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3b2dbe-5756-4458-bf3c-16b909c3da73 None admin] RBAC: Adding query filter params (type=network) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:44.187159 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3b2dbe-5756-4458-bf3c-16b909c3da73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:44.188595 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3b2dbe-5756-4458-bf3c-16b909c3da73 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:44.208822 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 390/783] 10.222.0.48 () {56 vars in 1112 bytes} [Fri Sep 9 20:47:43 2022] GET /identity/v3/services?type=network => generated 350 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:47:44.214441 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b2dbe-5756-4458-bf3c-16b909c3da73 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:44.314967 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b14b579-5a40-458a-9eb8-20c880ca2123 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:44.341817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:44.351074 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b2dbe-5756-4458-bf3c-16b909c3da73 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:44.376147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f317e1e-067a-45c4-bdea-a5f8a4c699b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:44.376942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f317e1e-067a-45c4-bdea-a5f8a4c699b8 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:44.377590 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f317e1e-067a-45c4-bdea-a5f8a4c699b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:44.377875 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f317e1e-067a-45c4-bdea-a5f8a4c699b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:44.378831 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f317e1e-067a-45c4-bdea-a5f8a4c699b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:44.379139 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f317e1e-067a-45c4-bdea-a5f8a4c699b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:44.380547 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f317e1e-067a-45c4-bdea-a5f8a4c699b8 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:44.385551 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410fa1c1-9cf2-49e2-8be1-a777bf622f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:44.386191 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fa1c1-9cf2-49e2-8be1-a777bf622f97 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:44.386471 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fa1c1-9cf2-49e2-8be1-a777bf622f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:44.386742 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fa1c1-9cf2-49e2-8be1-a777bf622f97 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:44.393142 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:44.411314 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 394/784] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:47:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:47:44.412401 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f317e1e-067a-45c4-bdea-a5f8a4c699b8 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:44.417017 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410fa1c1-9cf2-49e2-8be1-a777bf622f97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:47:44.431964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-410fa1c1-9cf2-49e2-8be1-a777bf622f97 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:44.431964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:44.431964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:44.431964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:44.431964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:44.431964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:44.431964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:44.431964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:44.431964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:44.431964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:44.431964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:44.431964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:44.431964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:47:44.431964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:47:44.431964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:47:44.431964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:47:44.432983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:47:44.432983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:47:44.432983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:47:44.432983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:47:44.432983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:44.432983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:44.432983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:44.432983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:44.432983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:44.432983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:44.432983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:44.432983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:44.432983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:44.432983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:44.432983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:44.432983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:44.433828 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:44.433828 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:44.433828 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:44.433828 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:44.433828 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:47:44.433828 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:47:44.433828 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:47:44.433828 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:47:44.433828 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:47:44.433828 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:47:44.433828 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:44.433828 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:44.438271 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 391/785] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Sep 9 20:47:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:44.439165 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410fa1c1-9cf2-49e2-8be1-a777bf622f97 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:44.535061 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f317e1e-067a-45c4-bdea-a5f8a4c699b8 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:44.575143 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1a44b-ec76-4326-a27b-868c1e3aaa30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:44.580093 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1a44b-ec76-4326-a27b-868c1e3aaa30 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:44.580093 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1a44b-ec76-4326-a27b-868c1e3aaa30 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:44.580093 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1a44b-ec76-4326-a27b-868c1e3aaa30 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:44.580093 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe1a44b-ec76-4326-a27b-868c1e3aaa30 None admin] RBAC: Adding query filter params (interface=public, service_id=791b089e453249328adfaefe883d93fd, region_id=RegionOne) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:44.580093 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe1a44b-ec76-4326-a27b-868c1e3aaa30 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:44.580093 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe1a44b-ec76-4326-a27b-868c1e3aaa30 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:44.585848 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 395/786] 10.222.0.48 () {56 vars in 1251 bytes} [Fri Sep 9 20:47:44 2022] GET /identity/v3/endpoints?service_id=791b089e453249328adfaefe883d93fd&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:44.586773 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1a44b-ec76-4326-a27b-868c1e3aaa30 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:44.636984 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410fa1c1-9cf2-49e2-8be1-a777bf622f97 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:44.742926 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6267711-f0dd-4cc2-83aa-ac3165c11a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:44.748093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6267711-f0dd-4cc2-83aa-ac3165c11a96 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:44.748093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6267711-f0dd-4cc2-83aa-ac3165c11a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:44.748093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6267711-f0dd-4cc2-83aa-ac3165c11a96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:44.775485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6267711-f0dd-4cc2-83aa-ac3165c11a96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:44.777253 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6267711-f0dd-4cc2-83aa-ac3165c11a96 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:44.808474 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-c6267711-f0dd-4cc2-83aa-ac3165c11a96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:44.808474 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:44.808474 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:44.808474 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:44.808474 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:44.808474 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:44.808474 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:44.808474 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:44.808474 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:44.808474 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:44.808474 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:44.808474 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:44.808474 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:44.808474 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:44.808474 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:44.808474 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:44.810978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:44.810978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:44.810978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:44.810978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:44.810978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:44.810978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:44.810978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:44.810978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:44.810978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:44.810978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:44.810978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:44.810978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:44.810978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:44.810978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:44.810978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:44.810978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:44.810978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:44.813186 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:44.813186 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:44.813186 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:44.819558 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 392/787] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:44.819558 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6267711-f0dd-4cc2-83aa-ac3165c11a96 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:44.822490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:44.836437 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:44.880553 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1a44b-ec76-4326-a27b-868c1e3aaa30 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:44.921347 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2c48e-e0dc-45ad-9f8f-5a665dc20e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:44.922309 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2c48e-e0dc-45ad-9f8f-5a665dc20e36 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:44.922831 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2c48e-e0dc-45ad-9f8f-5a665dc20e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:44.923386 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2c48e-e0dc-45ad-9f8f-5a665dc20e36 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:44.924808 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2c48e-e0dc-45ad-9f8f-5a665dc20e36 None admin] RBAC: Adding query filter params (name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:44.925323 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2c48e-e0dc-45ad-9f8f-5a665dc20e36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:44.927782 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2c48e-e0dc-45ad-9f8f-5a665dc20e36 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:44.944809 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 396/788] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Sep 9 20:47:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:44.944809 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2c48e-e0dc-45ad-9f8f-5a665dc20e36 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:44.958453 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6267711-f0dd-4cc2-83aa-ac3165c11a96 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:45.002045 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bdfa1-1cb7-49a5-ac65-d1b512d7f20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:45.003035 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bdfa1-1cb7-49a5-ac65-d1b512d7f20b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:45.003552 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bdfa1-1cb7-49a5-ac65-d1b512d7f20b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:45.004033 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bdfa1-1cb7-49a5-ac65-d1b512d7f20b None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:45.004884 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72bdfa1-1cb7-49a5-ac65-d1b512d7f20b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:45.006524 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72bdfa1-1cb7-49a5-ac65-d1b512d7f20b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:45.012714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 393/789] 10.222.0.48 () {56 vars in 1087 bytes} [Fri Sep 9 20:47:44 2022] GET /identity/v3/services => generated 772 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:45.014567 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bdfa1-1cb7-49a5-ac65-d1b512d7f20b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:45.054712 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2c48e-e0dc-45ad-9f8f-5a665dc20e36 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:45.080413 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:45.094920 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:45.095097 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26514f56-f674-4172-948f-5a8b8609eb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:45.095097 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26514f56-f674-4172-948f-5a8b8609eb8f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:45.095097 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26514f56-f674-4172-948f-5a8b8609eb8f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:45.095346 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26514f56-f674-4172-948f-5a8b8609eb8f None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:45.096003 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26514f56-f674-4172-948f-5a8b8609eb8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:45.096757 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26514f56-f674-4172-948f-5a8b8609eb8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:45.098464 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26514f56-f674-4172-948f-5a8b8609eb8f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:45.130754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 397/790] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:45.132249 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bdfa1-1cb7-49a5-ac65-d1b512d7f20b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:45.137065 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26514f56-f674-4172-948f-5a8b8609eb8f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:45.172435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c187d-88cd-4027-88e2-5e6e722ef03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:45.173383 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c187d-88cd-4027-88e2-5e6e722ef03f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:45.173915 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c187d-88cd-4027-88e2-5e6e722ef03f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:45.174326 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c187d-88cd-4027-88e2-5e6e722ef03f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:45.187589 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c187d-88cd-4027-88e2-5e6e722ef03f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:45.189363 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c187d-88cd-4027-88e2-5e6e722ef03f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:45.206151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-421c187d-88cd-4027-88e2-5e6e722ef03f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:45.206151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:45.206151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:45.206151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:45.206151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:45.206151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:45.206151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:45.206151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:45.206151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:45.206151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:45.206151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:45.206151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:45.206151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:45.206151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:45.206151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:45.206151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:45.207348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:45.207348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:45.207348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:45.207348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:45.207348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:45.207348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:45.207348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:45.207348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:45.207348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:45.207348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:45.207348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:45.207348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:45.207348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:45.207348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:45.207348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:45.207348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:45.207348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:45.208437 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:45.208437 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:45.208437 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:45.214659 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 394/791] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:45.216663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c187d-88cd-4027-88e2-5e6e722ef03f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:45.345371 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26514f56-f674-4172-948f-5a8b8609eb8f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:45.391314 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9544e696-7e5e-43f6-a010-de55c30c0e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:45.392300 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544e696-7e5e-43f6-a010-de55c30c0e5a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:45.392725 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544e696-7e5e-43f6-a010-de55c30c0e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:45.393138 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544e696-7e5e-43f6-a010-de55c30c0e5a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:45.402838 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9544e696-7e5e-43f6-a010-de55c30c0e5a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:45.404593 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9544e696-7e5e-43f6-a010-de55c30c0e5a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:45.417952 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-9544e696-7e5e-43f6-a010-de55c30c0e5a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 20:47:45.417952 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:45.417952 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:45.417952 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:45.417952 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:45.417952 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:45.417952 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:45.417952 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:45.417952 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:45.417952 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:45.417952 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:45.417952 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:45.417952 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:47:45.417952 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:47:45.417952 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:47:45.417952 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:47:45.419124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:45.419124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:45.419124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:47:45.419124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:45.419124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:47:45.419124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:45.419124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:45.419124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:45.419124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:45.419124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:45.419124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:45.419124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:45.419124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:45.419124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:45.419124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:45.419124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:45.419124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:45.419124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:45.420250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:45.420250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:45.420250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:47:45.420250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:47:45.420250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:47:45.420250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:47:45.420250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:47:45.420250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:47:45.420250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 20:47:45.420250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:45.422318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 398/792] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:47:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:47:45.429172 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9544e696-7e5e-43f6-a010-de55c30c0e5a None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:45.459267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c187d-88cd-4027-88e2-5e6e722ef03f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:45.482925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:45.494276 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:45.499801 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c7ce22-b71c-4ba6-b7ac-779c56a0b1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:45.506210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7ce22-b71c-4ba6-b7ac-779c56a0b1e1 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:45.506210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7ce22-b71c-4ba6-b7ac-779c56a0b1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:45.506210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7ce22-b71c-4ba6-b7ac-779c56a0b1e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:45.506210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c7ce22-b71c-4ba6-b7ac-779c56a0b1e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:45.506210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c7ce22-b71c-4ba6-b7ac-779c56a0b1e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:45.506210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c7ce22-b71c-4ba6-b7ac-779c56a0b1e1 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:45.519532 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 395/793] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:45.520865 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feafab6-2af6-4b8b-8c7c-d36531f51fe1 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:45.521154 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feafab6-2af6-4b8b-8c7c-d36531f51fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:45.521433 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feafab6-2af6-4b8b-8c7c-d36531f51fe1 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:45.523850 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 396/794] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:45.528419 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feafab6-2af6-4b8b-8c7c-d36531f51fe1 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:45.571352 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9544e696-7e5e-43f6-a010-de55c30c0e5a None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:45.610034 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f575c-f63b-425e-a92b-1c0abc1afc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:45.611066 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f575c-f63b-425e-a92b-1c0abc1afc0a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:45.611664 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f575c-f63b-425e-a92b-1c0abc1afc0a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:45.612209 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f575c-f63b-425e-a92b-1c0abc1afc0a None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:45.613170 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f575c-f63b-425e-a92b-1c0abc1afc0a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:45.613707 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f575c-f63b-425e-a92b-1c0abc1afc0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:45.615570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f575c-f63b-425e-a92b-1c0abc1afc0a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:45.641586 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 399/795] 10.222.0.48 () {56 vars in 1135 bytes} [Fri Sep 9 20:47:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:45.643954 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64d296-770c-4325-ac87-8ee972795cc5 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:45.644471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64d296-770c-4325-ac87-8ee972795cc5 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:45.644964 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64d296-770c-4325-ac87-8ee972795cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:45.646138 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feafab6-2af6-4b8b-8c7c-d36531f51fe1 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:45.667447 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-dc64d296-770c-4325-ac87-8ee972795cc5 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:45.669190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-dc64d296-770c-4325-ac87-8ee972795cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:45.683414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1415f12-1053-4007-8ee2-47f239422ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:45.684107 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1415f12-1053-4007-8ee2-47f239422ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:45.684385 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1415f12-1053-4007-8ee2-47f239422ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:45.684660 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1415f12-1053-4007-8ee2-47f239422ba1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:45.692633 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1415f12-1053-4007-8ee2-47f239422ba1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:45.694618 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1415f12-1053-4007-8ee2-47f239422ba1 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:45.707833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-c1415f12-1053-4007-8ee2-47f239422ba1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 20:47:45.707833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:45.707833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:45.707833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:45.707833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:45.707833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:45.707833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:45.707833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:45.707833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:45.707833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:45.707833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:45.707833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:45.707833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:47:45.707833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:47:45.707833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:47:45.707833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:47:45.708857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:45.708857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:45.708857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:47:45.708857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:45.708857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:47:45.708857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:45.708857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:45.708857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:45.708857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:45.708857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:45.708857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:45.708857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:45.708857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:45.708857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:45.708857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:45.708857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:45.708857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:45.708857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:45.710421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:45.710421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:45.710421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:47:45.710421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:47:45.710421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:47:45.710421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:47:45.710421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:47:45.710421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:47:45.710421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 20:47:45.710421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:45.712140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 397/796] 10.222.0.48 () {56 vars in 1117 bytes} [Fri Sep 9 20:47:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:45.713250 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1415f12-1053-4007-8ee2-47f239422ba1 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:45.765595 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 400/797] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:47:45.766463 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64d296-770c-4325-ac87-8ee972795cc5 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:45.781461 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:45.814195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:45.894544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1415f12-1053-4007-8ee2-47f239422ba1 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:45.948610 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64d296-770c-4325-ac87-8ee972795cc5 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:45.965325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df52c219-e672-4d09-8f6d-936681111608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:45.966291 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52c219-e672-4d09-8f6d-936681111608 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:45.966783 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52c219-e672-4d09-8f6d-936681111608 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:45.967283 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52c219-e672-4d09-8f6d-936681111608 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:45.982758 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df52c219-e672-4d09-8f6d-936681111608 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:45.985609 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df52c219-e672-4d09-8f6d-936681111608 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:46.014813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-df52c219-e672-4d09-8f6d-936681111608 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:46.014813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:46.014813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:46.014813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:46.014813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:46.014813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:46.014813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:46.014813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:46.014813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:46.014813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:46.014813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:46.014813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:46.014813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:46.014813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:46.014813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:46.014813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:46.015780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:46.015780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:46.015780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:46.015780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:46.015780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:46.015780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:46.015780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:46.015780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:46.015780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:46.015780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:46.015780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:46.015780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:46.015780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:46.015780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:46.015780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:46.015780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:46.015780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:46.017488 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:46.017488 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:46.017488 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:46.023933 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 398/798] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:46.025458 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00dd51-045b-43b5-8988-ac1ca01b6ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:46.026350 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00dd51-045b-43b5-8988-ac1ca01b6ff1 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:46.026922 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00dd51-045b-43b5-8988-ac1ca01b6ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:46.027429 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00dd51-045b-43b5-8988-ac1ca01b6ff1 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:46.028416 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e00dd51-045b-43b5-8988-ac1ca01b6ff1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:46.029452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df52c219-e672-4d09-8f6d-936681111608 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:46.029971 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e00dd51-045b-43b5-8988-ac1ca01b6ff1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:46.031978 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e00dd51-045b-43b5-8988-ac1ca01b6ff1 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:46.072280 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 401/799] 10.222.0.48 () {56 vars in 1149 bytes} [Fri Sep 9 20:47:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:46.077654 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00dd51-045b-43b5-8988-ac1ca01b6ff1 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:46.170748 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:46.183112 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df52c219-e672-4d09-8f6d-936681111608 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:46.197546 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:46.211535 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00dd51-045b-43b5-8988-ac1ca01b6ff1 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:46.225153 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273b29e-7384-465a-bed4-2d0960a64d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:46.225781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273b29e-7384-465a-bed4-2d0960a64d87 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:46.226065 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273b29e-7384-465a-bed4-2d0960a64d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:46.226336 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273b29e-7384-465a-bed4-2d0960a64d87 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:46.227010 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273b29e-7384-465a-bed4-2d0960a64d87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:46.227320 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273b29e-7384-465a-bed4-2d0960a64d87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:46.228785 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273b29e-7384-465a-bed4-2d0960a64d87 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:46.242489 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 399/800] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:46.243356 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273b29e-7384-465a-bed4-2d0960a64d87 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:46.252916 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9deb49-f6d0-4b84-81f4-737d72284ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:46.253691 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9deb49-f6d0-4b84-81f4-737d72284ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:46.253974 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9deb49-f6d0-4b84-81f4-737d72284ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:46.254244 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9deb49-f6d0-4b84-81f4-737d72284ea1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:46.265184 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9deb49-f6d0-4b84-81f4-737d72284ea1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:46.266615 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9deb49-f6d0-4b84-81f4-737d72284ea1 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:46.281078 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-3d9deb49-f6d0-4b84-81f4-737d72284ea1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:46.281078 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:46.281078 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:46.281078 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:46.281078 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:46.281078 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:46.281078 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:46.281078 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:46.281078 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:46.281078 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:46.281078 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:46.281078 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:46.281078 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:46.281078 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:46.281078 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:46.281078 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:46.282046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:46.282046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:46.282046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:46.282046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:46.282046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:46.282046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:46.282046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:46.282046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:46.282046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:46.282046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:46.282046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:46.282046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:46.282046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:46.282046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:46.282046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:46.282046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:46.282046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:46.282950 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:46.282950 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:46.282950 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:46.282950 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 402/801] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:46.283784 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08b88f-2a76-4634-ae6d-22ff5ba4cb90 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:46.284091 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08b88f-2a76-4634-ae6d-22ff5ba4cb90 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:46.284370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08b88f-2a76-4634-ae6d-22ff5ba4cb90 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:46.288208 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 403/802] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:46.288208 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08b88f-2a76-4634-ae6d-22ff5ba4cb90 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:46.356637 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273b29e-7384-465a-bed4-2d0960a64d87 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:46.392699 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631cf697-03e3-467b-ac2e-0a62145215a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:46.393323 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631cf697-03e3-467b-ac2e-0a62145215a5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:46.393600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631cf697-03e3-467b-ac2e-0a62145215a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:46.393871 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631cf697-03e3-467b-ac2e-0a62145215a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:46.395403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08b88f-2a76-4634-ae6d-22ff5ba4cb90 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:46.403950 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631cf697-03e3-467b-ac2e-0a62145215a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:47:46.408883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-631cf697-03e3-467b-ac2e-0a62145215a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:47:46.408883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:46.408883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:46.408883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:46.408883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:46.408883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:46.408883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:46.408883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:46.408883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:46.408883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:46.408883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:46.408883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:46.408883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:47:46.408883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:47:46.408883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:47:46.408883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:47:46.409811 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:47:46.409811 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:47:46.409811 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:47:46.409811 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:47:46.409811 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:46.409811 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:46.409811 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:46.409811 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:46.409811 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:46.409811 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:46.409811 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:46.409811 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:46.409811 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:46.409811 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:46.409811 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:46.409811 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:46.410658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:46.410658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:46.410658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:46.410658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:46.410658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:47:46.410658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:47:46.410658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:47:46.410658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:47:46.410658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:47:46.410658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:47:46.410658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:47:46.410658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:46.410658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 400/803] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:47:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:46.412001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631cf697-03e3-467b-ac2e-0a62145215a5 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:46.429708 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:46.436827 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778043b-28b4-4aea-87e8-d298ed4e2639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:46.437462 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778043b-28b4-4aea-87e8-d298ed4e2639 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:46.437741 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778043b-28b4-4aea-87e8-d298ed4e2639 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:46.438117 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778043b-28b4-4aea-87e8-d298ed4e2639 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:46.445368 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7778043b-28b4-4aea-87e8-d298ed4e2639 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:46.446799 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7778043b-28b4-4aea-87e8-d298ed4e2639 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:46.450256 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:46.459569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-7778043b-28b4-4aea-87e8-d298ed4e2639 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:46.459569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:46.459569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:46.459569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:46.459569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:46.459569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:46.459569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:46.459569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:46.459569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:46.459569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:46.459569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:46.459569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:46.459569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:47:46.459569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:47:46.459569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:47:46.460496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:47:46.460496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:46.460496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:46.460496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:46.460496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:46.460496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:46.460496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:46.460496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:46.460496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:46.460496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:46.460496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:46.460496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:46.460496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:46.460496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:46.460496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:46.460496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:46.460496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:46.461951 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:47:46.461951 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:47:46.461951 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:47:46.461951 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:47:46.461951 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:47:46.461951 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:47:46.461951 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:46.461951 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:46.461951 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 404/804] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:47:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:46.462446 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a123a5-efaf-4121-949a-104832b0c089 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:46.462515 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a123a5-efaf-4121-949a-104832b0c089 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:46.462775 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a123a5-efaf-4121-949a-104832b0c089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:46.481220 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-e2a123a5-efaf-4121-949a-104832b0c089 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:46.482422 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e2a123a5-efaf-4121-949a-104832b0c089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:46.519377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631cf697-03e3-467b-ac2e-0a62145215a5 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:46.554000 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eedec6-7d17-489e-80c2-2e9fbea250c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:46.554745 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eedec6-7d17-489e-80c2-2e9fbea250c1 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:46.555022 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eedec6-7d17-489e-80c2-2e9fbea250c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:46.555297 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 405/805] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:47:46.555404 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eedec6-7d17-489e-80c2-2e9fbea250c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:46.556013 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eedec6-7d17-489e-80c2-2e9fbea250c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:46.556163 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a123a5-efaf-4121-949a-104832b0c089 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:46.556346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eedec6-7d17-489e-80c2-2e9fbea250c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:46.557790 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eedec6-7d17-489e-80c2-2e9fbea250c1 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:46.574180 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 401/806] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:46.575097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eedec6-7d17-489e-80c2-2e9fbea250c1 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:46.668612 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a123a5-efaf-4121-949a-104832b0c089 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:46.669142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:46.702989 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eedec6-7d17-489e-80c2-2e9fbea250c1 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:46.711968 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:46.718182 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbadbf8-7718-4d41-9372-334e853d4814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:46.718791 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbadbf8-7718-4d41-9372-334e853d4814 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:46.719072 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbadbf8-7718-4d41-9372-334e853d4814 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:46.719347 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbadbf8-7718-4d41-9372-334e853d4814 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:46.720008 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbadbf8-7718-4d41-9372-334e853d4814 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:46.720339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbadbf8-7718-4d41-9372-334e853d4814 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:46.721884 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbadbf8-7718-4d41-9372-334e853d4814 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:46.736296 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 406/807] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:47:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:46.737896 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbadbf8-7718-4d41-9372-334e853d4814 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:46.747676 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8671c23c-da19-455f-a5c7-6cfa5e556998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:46.748317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671c23c-da19-455f-a5c7-6cfa5e556998 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:46.748597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671c23c-da19-455f-a5c7-6cfa5e556998 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:46.748868 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671c23c-da19-455f-a5c7-6cfa5e556998 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:46.749984 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8671c23c-da19-455f-a5c7-6cfa5e556998 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:46.750299 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8671c23c-da19-455f-a5c7-6cfa5e556998 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:46.751791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8671c23c-da19-455f-a5c7-6cfa5e556998 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:46.768538 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 402/808] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:47:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:46.770381 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8671c23c-da19-455f-a5c7-6cfa5e556998 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:46.864855 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbadbf8-7718-4d41-9372-334e853d4814 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:46.875805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8671c23c-da19-455f-a5c7-6cfa5e556998 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:46.900397 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0f29c-7385-4e91-b2b4-18ec7e29c8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:46.901038 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0f29c-7385-4e91-b2b4-18ec7e29c8b5 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:46.901322 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0f29c-7385-4e91-b2b4-18ec7e29c8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:46.901596 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0f29c-7385-4e91-b2b4-18ec7e29c8b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:46.906953 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:46.908547 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad0f29c-7385-4e91-b2b4-18ec7e29c8b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:46.910291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad0f29c-7385-4e91-b2b4-18ec7e29c8b5 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:46.912295 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bfac83-a723-416a-bcaa-d252a754c1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:46.916487 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfac83-a723-416a-bcaa-d252a754c1c8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:46.916487 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfac83-a723-416a-bcaa-d252a754c1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:46.916487 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfac83-a723-416a-bcaa-d252a754c1c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:46.923148 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-8ad0f29c-7385-4e91-b2b4-18ec7e29c8b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:46.923148 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:46.923148 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:46.923148 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:46.923148 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:46.923148 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:46.923148 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:46.923148 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:46.923148 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:46.923148 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:46.923148 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:46.923148 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:46.923148 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:47:46.923148 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:47:46.923148 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:47:46.924186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:47:46.924186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:46.924186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:46.924186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:46.924186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:46.924186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:46.924186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:46.924186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:46.924186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:46.924186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:46.924186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:46.924186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:46.924186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:46.924186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:46.924186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:46.924186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:46.924186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:46.925119 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:47:46.925119 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:47:46.925119 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:47:46.925119 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:47:46.925119 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:47:46.925119 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:47:46.925119 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:46.925119 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:46.925119 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 407/809] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:47:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:46.925119 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0f29c-7385-4e91-b2b4-18ec7e29c8b5 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:46.930018 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bfac83-a723-416a-bcaa-d252a754c1c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:46.931531 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bfac83-a723-416a-bcaa-d252a754c1c8 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:46.949402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-33bfac83-a723-416a-bcaa-d252a754c1c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:46.949402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:46.949402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:46.949402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:46.949402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:46.949402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:46.949402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:46.949402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:46.949402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:46.949402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:46.949402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:46.949402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:46.949402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:46.949402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:46.949402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:46.949402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:46.950397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:46.950397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:46.950397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:46.950397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:46.950397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:46.950397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:46.950397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:46.950397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:46.950397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:46.950397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:46.950397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:46.950397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:46.950397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:46.950397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:46.950397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:46.950397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:46.950397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:46.951294 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:46.951294 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:46.951294 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:46.951294 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 403/810] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:46.951294 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bfac83-a723-416a-bcaa-d252a754c1c8 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:46.990010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:47.042811 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0f29c-7385-4e91-b2b4-18ec7e29c8b5 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:47.058767 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bfac83-a723-416a-bcaa-d252a754c1c8 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:47.079551 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251df26-bd4d-4ddf-b5e7-f4233e2d3e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:47.080254 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251df26-bd4d-4ddf-b5e7-f4233e2d3e65 None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:47.080537 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251df26-bd4d-4ddf-b5e7-f4233e2d3e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:47.080830 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251df26-bd4d-4ddf-b5e7-f4233e2d3e65 None admin] PATH_INFO: `/v3/projects/91319aa782334fb9930770029b74e84a/users/7d28fa5d49d44120bd5eaec2d4e5fdbd/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:47.082698 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c251df26-bd4d-4ddf-b5e7-f4233e2d3e65 None admin] RBAC: Authorizing `identity:create_grant(project_id=91319aa782334fb9930770029b74e84a, user_id=7d28fa5d49d44120bd5eaec2d4e5fdbd, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:47.084155 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c251df26-bd4d-4ddf-b5e7-f4233e2d3e65 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:47.094590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bff393-48a7-4e6b-852a-903b3a7b4aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:47.095223 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bff393-48a7-4e6b-852a-903b3a7b4aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:47.095545 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bff393-48a7-4e6b-852a-903b3a7b4aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:47.095779 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bff393-48a7-4e6b-852a-903b3a7b4aa5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:47.096396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bff393-48a7-4e6b-852a-903b3a7b4aa5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:47.097756 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bff393-48a7-4e6b-852a-903b3a7b4aa5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:47.107018 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-06bff393-48a7-4e6b-852a-903b3a7b4aa5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 20:47:47.107018 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:47.107018 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:47:47.107018 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:47:47.107018 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:47:47.107018 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:47:47.107018 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:47:47.107018 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:47:47.107018 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 20:47:47.107018 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:47.107018 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:47:47.107018 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:47.107018 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:47.107018 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:47.107018 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:47.107018 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:47.107018 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:47.108299 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:47.108299 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:47.108299 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:47.108299 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:47.108299 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:47.108299 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:47.108299 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:47:47.108299 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:47:47.108299 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:47:47.108299 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:47:47.108299 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:47.108299 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:47.108299 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:47.108299 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:47.108299 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:47.108299 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:47.109168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:47.109168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:47.109168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:47.109168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:47.109168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:47.109168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:47.109168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:47.109168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:47.109168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:47.109168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:47.109168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:47:47.109168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:47:47.109168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 20:47:47.109168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:47.109168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 404/811] 10.222.0.48 () {56 vars in 1129 bytes} [Fri Sep 9 20:47:46 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:47.109966 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bff393-48a7-4e6b-852a-903b3a7b4aa5 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:47.154910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 408/812] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Sep 9 20:47:46 2022] PUT /identity/v3/projects/91319aa782334fb9930770029b74e84a/users/7d28fa5d49d44120bd5eaec2d4e5fdbd/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:47:47.165755 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251df26-bd4d-4ddf-b5e7-f4233e2d3e65 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:47.168041 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:47.244634 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bff393-48a7-4e6b-852a-903b3a7b4aa5 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:47.283282 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f62f89-ac48-4c20-9137-06970517f88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:47.283938 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f62f89-ac48-4c20-9137-06970517f88d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:47.284263 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f62f89-ac48-4c20-9137-06970517f88d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:47.284618 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f62f89-ac48-4c20-9137-06970517f88d None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:47.285248 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f62f89-ac48-4c20-9137-06970517f88d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:47.285557 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f62f89-ac48-4c20-9137-06970517f88d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:47.287131 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f62f89-ac48-4c20-9137-06970517f88d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:47.305817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 405/813] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:47:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:47.305817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f62f89-ac48-4c20-9137-06970517f88d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:47.341159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251df26-bd4d-4ddf-b5e7-f4233e2d3e65 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:47.358341 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:47.387268 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764454b-46e8-4e9e-b045-c5c5ddaf8f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:47.388172 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764454b-46e8-4e9e-b045-c5c5ddaf8f42 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:47.390514 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764454b-46e8-4e9e-b045-c5c5ddaf8f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:47.391071 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764454b-46e8-4e9e-b045-c5c5ddaf8f42 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:47.391737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764454b-46e8-4e9e-b045-c5c5ddaf8f42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:47.393505 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764454b-46e8-4e9e-b045-c5c5ddaf8f42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:47.394904 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764454b-46e8-4e9e-b045-c5c5ddaf8f42 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:47.409400 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 409/814] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:47.410370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764454b-46e8-4e9e-b045-c5c5ddaf8f42 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:47.435407 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f62f89-ac48-4c20-9137-06970517f88d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:47.446994 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:47.473649 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f064a8-9818-4097-9412-b04a4ae90ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:47.474515 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f064a8-9818-4097-9412-b04a4ae90ddb None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:47.474930 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f064a8-9818-4097-9412-b04a4ae90ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:47.475310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f064a8-9818-4097-9412-b04a4ae90ddb None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:47.476187 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f064a8-9818-4097-9412-b04a4ae90ddb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:47.476589 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f064a8-9818-4097-9412-b04a4ae90ddb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:47.478240 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f064a8-9818-4097-9412-b04a4ae90ddb None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:47.488756 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 406/815] 10.222.0.48 () {56 vars in 1124 bytes} [Fri Sep 9 20:47:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:47.489851 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f064a8-9818-4097-9412-b04a4ae90ddb None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:47.540356 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764454b-46e8-4e9e-b045-c5c5ddaf8f42 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:47.580460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5046c75c-dab0-49d7-a031-9b5c215c10e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:47.581151 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046c75c-dab0-49d7-a031-9b5c215c10e0 None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:47.581427 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046c75c-dab0-49d7-a031-9b5c215c10e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:47.581694 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046c75c-dab0-49d7-a031-9b5c215c10e0 None admin] PATH_INFO: `/v3/projects/91319aa782334fb9930770029b74e84a/users/8c21ea7d3cbb494886e60a4bd4f39145/roles/1e000597af164d54a40df2a39c729820` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:47.596987 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5046c75c-dab0-49d7-a031-9b5c215c10e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=91319aa782334fb9930770029b74e84a, user_id=8c21ea7d3cbb494886e60a4bd4f39145, role_id=1e000597af164d54a40df2a39c729820)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:47.598473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5046c75c-dab0-49d7-a031-9b5c215c10e0 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:47.601732 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f064a8-9818-4097-9412-b04a4ae90ddb None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:47.635960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807bbfb-dd23-42d4-81a5-df76e1611d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:47.636623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807bbfb-dd23-42d4-81a5-df76e1611d43 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:47.636906 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807bbfb-dd23-42d4-81a5-df76e1611d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:47.637175 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807bbfb-dd23-42d4-81a5-df76e1611d43 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:47.643994 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807bbfb-dd23-42d4-81a5-df76e1611d43 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:47.645456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807bbfb-dd23-42d4-81a5-df76e1611d43 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:47.656953 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-6807bbfb-dd23-42d4-81a5-df76e1611d43 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:47:47.656953 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:47.656953 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:47.656953 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:47.656953 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:47.656953 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:47.656953 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:47.656953 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:47.656953 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:47.656953 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:47.656953 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:47.656953 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:47.656953 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:47:47.656953 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:47:47.656953 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:47:47.656953 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:47:47.657954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:47.657954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:47.657954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:47:47.657954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:47.657954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:47:47.657954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:47.657954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:47.657954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:47.657954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:47.657954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:47.657954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:47.657954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:47.657954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:47.657954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:47.657954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:47.657954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:47.657954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:47.657954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:47.658889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:47.658889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:47.658889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:47:47.658889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:47:47.658889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:47:47.658889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:47:47.658889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:47:47.658889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:47:47.658889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:47:47.658889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:47.658889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 407/816] 10.222.0.48 () {56 vars in 1093 bytes} [Fri Sep 9 20:47:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:47.658889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807bbfb-dd23-42d4-81a5-df76e1611d43 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:47.704711 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 410/817] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Sep 9 20:47:47 2022] PUT /identity/v3/projects/91319aa782334fb9930770029b74e84a/users/8c21ea7d3cbb494886e60a4bd4f39145/roles/1e000597af164d54a40df2a39c729820 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:47:47.705822 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:47.707046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb9758-33f7-46e5-9cb9-93e657e5ffb8 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:47.707388 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb9758-33f7-46e5-9cb9-93e657e5ffb8 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:47.707719 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb9758-33f7-46e5-9cb9-93e657e5ffb8 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:47.713432 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 411/818] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:47.713432 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb9758-33f7-46e5-9cb9-93e657e5ffb8 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:47.748359 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:47.811890 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807bbfb-dd23-42d4-81a5-df76e1611d43 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:47.844309 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb9758-33f7-46e5-9cb9-93e657e5ffb8 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:47.862174 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6d346-155c-400a-ac52-c0eb84aed5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:47.863330 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6d346-155c-400a-ac52-c0eb84aed5e3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:47.863755 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6d346-155c-400a-ac52-c0eb84aed5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:47.864195 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6d346-155c-400a-ac52-c0eb84aed5e3 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:47.865297 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd6d346-155c-400a-ac52-c0eb84aed5e3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:47.866699 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd6d346-155c-400a-ac52-c0eb84aed5e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:47.868259 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd6d346-155c-400a-ac52-c0eb84aed5e3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:47.875732 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 408/819] 10.222.0.48 () {56 vars in 1124 bytes} [Fri Sep 9 20:47:47 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:47.877352 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c690e8-ca1f-41f3-8646-f3284052719a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:47.877993 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c690e8-ca1f-41f3-8646-f3284052719a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:47.878413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c690e8-ca1f-41f3-8646-f3284052719a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:47.892335 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4844a0ec-c471-49f0-907b-b0e7d12ae778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:47.893088 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844a0ec-c471-49f0-907b-b0e7d12ae778 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:47.893457 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844a0ec-c471-49f0-907b-b0e7d12ae778 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:47.893822 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844a0ec-c471-49f0-907b-b0e7d12ae778 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:47.894652 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4844a0ec-c471-49f0-907b-b0e7d12ae778 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:47.895088 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4844a0ec-c471-49f0-907b-b0e7d12ae778 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:47.897001 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4844a0ec-c471-49f0-907b-b0e7d12ae778 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:47.901007 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b3c690e8-ca1f-41f3-8646-f3284052719a None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:47.901501 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b3c690e8-ca1f-41f3-8646-f3284052719a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:47.923151 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 412/820] 10.222.0.48 () {56 vars in 1133 bytes} [Fri Sep 9 20:47:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:47.923151 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4844a0ec-c471-49f0-907b-b0e7d12ae778 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:48.001398 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 409/821] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:47:48.002406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c690e8-ca1f-41f3-8646-f3284052719a None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:48.024430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:48.044611 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:48.067921 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4844a0ec-c471-49f0-907b-b0e7d12ae778 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:48.141488 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5512c-30d2-49c0-9ecd-36248dba2cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:48.142400 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5512c-30d2-49c0-9ecd-36248dba2cea None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:48.143833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5512c-30d2-49c0-9ecd-36248dba2cea None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:48.144786 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5512c-30d2-49c0-9ecd-36248dba2cea None admin] PATH_INFO: `/v3/endpoints` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:48.145751 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c5512c-30d2-49c0-9ecd-36248dba2cea None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:48.149138 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c690e8-ca1f-41f3-8646-f3284052719a None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:48.150169 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c5512c-30d2-49c0-9ecd-36248dba2cea None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:48.198706 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b480ee-025b-4819-819b-f10daa370b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:48.199519 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b480ee-025b-4819-819b-f10daa370b25 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:48.199893 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b480ee-025b-4819-819b-f10daa370b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:48.200289 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b480ee-025b-4819-819b-f10daa370b25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:48.274245 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 413/822] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 20:47:47 2022] POST /identity/v3/endpoints => generated 346 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:47:48.287378 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b480ee-025b-4819-819b-f10daa370b25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:48.287378 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b480ee-025b-4819-819b-f10daa370b25 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:48.322379 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:48.324533 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5512c-30d2-49c0-9ecd-36248dba2cea None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:48.432027 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 410/823] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:47:48 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:47:48.437517 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b480ee-025b-4819-819b-f10daa370b25 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:48.479692 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5512c-30d2-49c0-9ecd-36248dba2cea None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:48.519608 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:48.529629 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc7c4d-d163-464d-b61d-41f8429f996f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:48.530632 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc7c4d-d163-464d-b61d-41f8429f996f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:48.531250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc7c4d-d163-464d-b61d-41f8429f996f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:48.532516 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc7c4d-d163-464d-b61d-41f8429f996f None admin] PATH_INFO: `/v3/services/network` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:48.533486 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc7c4d-d163-464d-b61d-41f8429f996f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:48.536655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc7c4d-d163-464d-b61d-41f8429f996f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:48.554793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-74bc7c4d-d163-464d-b61d-41f8429f996f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 20:47:48.554793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:48.554793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:47:48.554793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:47:48.554793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:47:48.554793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:47:48.554793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:47:48.554793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:47:48.554793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 20:47:48.554793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:48.554793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:47:48.554793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:48.554793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:48.554793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:48.554793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:48.554793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:48.554793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:48.556140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:48.556140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:48.556140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:48.556140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:48.556140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:48.556140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:48.556140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:47:48.556140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:47:48.556140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:47:48.556140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:47:48.556140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:48.556140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:48.556140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:48.556140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:48.556140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:48.556140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:48.557050 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:48.557050 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:48.557050 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:48.557050 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:48.557050 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:48.557050 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:48.557050 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:48.557050 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:48.557050 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:48.557050 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:48.557050 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:47:48.557050 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:47:48.557050 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 20:47:48.557050 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:48.559963 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 414/824] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:47:48 2022] GET /identity/v3/services/network => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:48.569153 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc7c4d-d163-464d-b61d-41f8429f996f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:48.612984 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b480ee-025b-4819-819b-f10daa370b25 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:48.644044 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:48.653262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324d681-5c31-4cf3-8d7c-5fff68acec40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:48.653937 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324d681-5c31-4cf3-8d7c-5fff68acec40 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:48.654934 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324d681-5c31-4cf3-8d7c-5fff68acec40 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:48.655212 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324d681-5c31-4cf3-8d7c-5fff68acec40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:48.665469 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324d681-5c31-4cf3-8d7c-5fff68acec40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:48.666928 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324d681-5c31-4cf3-8d7c-5fff68acec40 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:48.683680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-8324d681-5c31-4cf3-8d7c-5fff68acec40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:48.683680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:48.683680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:48.683680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:48.683680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:48.683680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:48.683680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:48.683680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:48.683680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:48.683680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:48.683680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:48.683680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:48.683680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:48.683680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:48.683680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:48.683680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:48.684747 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:48.684747 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:48.684747 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:48.684747 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:48.684747 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:48.684747 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:48.684747 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:48.684747 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:48.684747 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:48.684747 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:48.684747 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:48.684747 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:48.684747 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:48.684747 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:48.684747 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:48.684747 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:48.684747 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:48.685622 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:48.685622 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:48.685622 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:48.686093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 411/825] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:48.691287 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324d681-5c31-4cf3-8d7c-5fff68acec40 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:48.700569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc7c4d-d163-464d-b61d-41f8429f996f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:48.740402 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46568a1a-4089-4bae-b931-199f3c8e1976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:48.741194 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46568a1a-4089-4bae-b931-199f3c8e1976 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:48.741578 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46568a1a-4089-4bae-b931-199f3c8e1976 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:48.741962 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46568a1a-4089-4bae-b931-199f3c8e1976 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:48.742735 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46568a1a-4089-4bae-b931-199f3c8e1976 None admin] RBAC: Adding query filter params (name=network) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:48.743443 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46568a1a-4089-4bae-b931-199f3c8e1976 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:48.744836 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46568a1a-4089-4bae-b931-199f3c8e1976 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:48.750252 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 415/826] 10.222.0.48 () {56 vars in 1112 bytes} [Fri Sep 9 20:47:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:48.755700 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46568a1a-4089-4bae-b931-199f3c8e1976 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:48.856446 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:48.858887 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324d681-5c31-4cf3-8d7c-5fff68acec40 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:48.910681 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7be44-28f9-49fd-98e3-d8f56acfeeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:48.911569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7be44-28f9-49fd-98e3-d8f56acfeeb8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:48.911982 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7be44-28f9-49fd-98e3-d8f56acfeeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:48.912616 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7be44-28f9-49fd-98e3-d8f56acfeeb8 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:48.913548 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc7be44-28f9-49fd-98e3-d8f56acfeeb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:48.914015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc7be44-28f9-49fd-98e3-d8f56acfeeb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:48.915769 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc7be44-28f9-49fd-98e3-d8f56acfeeb8 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:48.930739 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 412/827] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:47:48.937024 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7be44-28f9-49fd-98e3-d8f56acfeeb8 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:48.948970 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46568a1a-4089-4bae-b931-199f3c8e1976 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:49.010517 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:49.020676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c7b349-428d-496b-9f19-5b2a6cc46989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:49.021544 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7b349-428d-496b-9f19-5b2a6cc46989 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:49.022007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7b349-428d-496b-9f19-5b2a6cc46989 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:49.023760 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7b349-428d-496b-9f19-5b2a6cc46989 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:49.024590 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c7b349-428d-496b-9f19-5b2a6cc46989 None admin] RBAC: Adding query filter params (type=network) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:49.026085 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c7b349-428d-496b-9f19-5b2a6cc46989 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:49.027444 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c7b349-428d-496b-9f19-5b2a6cc46989 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:49.044185 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 416/828] 10.222.0.48 () {56 vars in 1112 bytes} [Fri Sep 9 20:47:48 2022] GET /identity/v3/services?type=network => generated 350 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:49.048290 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c7b349-428d-496b-9f19-5b2a6cc46989 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:49.127979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7be44-28f9-49fd-98e3-d8f56acfeeb8 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:49.166462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb103d9-f367-47d5-8519-d10709f2f79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:49.167104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb103d9-f367-47d5-8519-d10709f2f79a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:49.167382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb103d9-f367-47d5-8519-d10709f2f79a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:49.167649 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb103d9-f367-47d5-8519-d10709f2f79a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:49.174337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb103d9-f367-47d5-8519-d10709f2f79a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:49.175890 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb103d9-f367-47d5-8519-d10709f2f79a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:49.188338 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-beb103d9-f367-47d5-8519-d10709f2f79a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:49.188338 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:49.188338 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:49.188338 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:49.188338 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:49.188338 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:49.188338 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:49.188338 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:49.188338 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:49.188338 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:49.188338 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:49.188338 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:49.188338 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:47:49.188338 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:47:49.188338 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:47:49.189251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:47:49.189251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:49.189251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:49.189251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:49.189251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:49.189251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:49.189251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:49.189251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:49.189251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:49.189251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:49.189251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:49.189251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:49.189251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:49.189251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:49.189251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:49.189251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:49.189251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:49.190213 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:47:49.190213 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:47:49.190213 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:47:49.190213 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:47:49.190213 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:47:49.190213 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:47:49.190213 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:49.190213 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:49.194702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 413/829] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:47:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:49.200095 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb103d9-f367-47d5-8519-d10709f2f79a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:49.239058 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c7b349-428d-496b-9f19-5b2a6cc46989 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:49.296465 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:49.323497 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05744f34-9920-4a67-8bc1-b796f45c481f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:49.324156 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05744f34-9920-4a67-8bc1-b796f45c481f None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:49.324434 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05744f34-9920-4a67-8bc1-b796f45c481f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:49.324702 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05744f34-9920-4a67-8bc1-b796f45c481f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:49.326590 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05744f34-9920-4a67-8bc1-b796f45c481f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:49.326870 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05744f34-9920-4a67-8bc1-b796f45c481f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:49.362177 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:49.366717 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb103d9-f367-47d5-8519-d10709f2f79a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:49.401579 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c3e86-0fa8-47ad-a9e0-745305f2aaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:49.402240 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c3e86-0fa8-47ad-a9e0-745305f2aaec None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:49.402547 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c3e86-0fa8-47ad-a9e0-745305f2aaec None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:49.402821 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c3e86-0fa8-47ad-a9e0-745305f2aaec None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:49.403511 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c3e86-0fa8-47ad-a9e0-745305f2aaec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:49.403853 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c3e86-0fa8-47ad-a9e0-745305f2aaec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:49.405456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c3e86-0fa8-47ad-a9e0-745305f2aaec None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:49.418638 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 414/830] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:47:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:49.424693 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c3e86-0fa8-47ad-a9e0-745305f2aaec None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:49.439968 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 417/831] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 20:47:49 2022] POST /identity/v3/endpoints => generated 337 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:47:49.441590 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0101f4d-9b9e-4b95-9354-11e38e0487ad None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:49.449331 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0101f4d-9b9e-4b95-9354-11e38e0487ad None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:49.449331 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0101f4d-9b9e-4b95-9354-11e38e0487ad None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:49.449331 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 418/832] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:49.453488 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca07a62-870c-4e94-8670-b4a849521604 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:49.454046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca07a62-870c-4e94-8670-b4a849521604 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:49.454462 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca07a62-870c-4e94-8670-b4a849521604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:49.488047 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-0ca07a62-870c-4e94-8670-b4a849521604 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:49.491937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0ca07a62-870c-4e94-8670-b4a849521604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:49.634420 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c3e86-0fa8-47ad-a9e0-745305f2aaec None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:49.645216 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 419/833] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:49 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:47:49.682052 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22836848-41bf-4390-93da-6844d7b715b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:49.682894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22836848-41bf-4390-93da-6844d7b715b1 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:49.683395 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22836848-41bf-4390-93da-6844d7b715b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:49.683830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22836848-41bf-4390-93da-6844d7b715b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:49.684771 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22836848-41bf-4390-93da-6844d7b715b1 None admin] RBAC: Adding query filter params (role.id=fcbf519fbc84458f913671a5a1d0914c, scope.project.id=91319aa782334fb9930770029b74e84a, user.id=613b845a9b4d432daf8d478fc3970708) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:49.685271 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22836848-41bf-4390-93da-6844d7b715b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:49.687105 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22836848-41bf-4390-93da-6844d7b715b1 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:49.700215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 415/834] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:47:49 2022] GET /identity/v3/role_assignments?user.id=613b845a9b4d432daf8d478fc3970708&scope.project.id=91319aa782334fb9930770029b74e84a&role.id=fcbf519fbc84458f913671a5a1d0914c => generated 608 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:49.952196 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca07a62-870c-4e94-8670-b4a849521604 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:50.055844 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca07a62-870c-4e94-8670-b4a849521604 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:50.063578 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:50.090878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3269dd14-d600-4003-a867-2d1b6c17a60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:50.091565 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269dd14-d600-4003-a867-2d1b6c17a60a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:50.091845 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269dd14-d600-4003-a867-2d1b6c17a60a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:50.092189 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269dd14-d600-4003-a867-2d1b6c17a60a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:50.103657 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3269dd14-d600-4003-a867-2d1b6c17a60a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:47:50.108833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-3269dd14-d600-4003-a867-2d1b6c17a60a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:47:50.108833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:50.108833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:50.108833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:50.108833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:50.108833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:50.108833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:50.108833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:50.108833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:50.108833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:50.108833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:50.108833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:50.108833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:47:50.108833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:47:50.108833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:47:50.108833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:47:50.109850 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:47:50.109850 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:47:50.109850 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:47:50.109850 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:47:50.109850 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:50.109850 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:50.109850 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:50.109850 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:50.109850 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:50.109850 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:50.109850 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:50.109850 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:50.109850 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:50.109850 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:50.109850 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:50.109850 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:50.110708 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:50.110708 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:50.110708 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:50.110708 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:50.110708 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:47:50.110708 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:47:50.110708 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:47:50.110708 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:47:50.110708 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:47:50.110708 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:47:50.110708 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:47:50.110708 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:50.110708 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 420/835] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:47:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:50.116635 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22836848-41bf-4390-93da-6844d7b715b1 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:50.145225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:50.222459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22836848-41bf-4390-93da-6844d7b715b1 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:50.228576 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195791c7-f7f7-447a-a9da-8dc58ed71bcd None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:50.228870 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195791c7-f7f7-447a-a9da-8dc58ed71bcd None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:50.229154 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195791c7-f7f7-447a-a9da-8dc58ed71bcd None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:50.230496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 421/836] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:50.235655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da9111-149f-4474-8b32-4aa676155d54 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:50.235941 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da9111-149f-4474-8b32-4aa676155d54 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:50.236252 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da9111-149f-4474-8b32-4aa676155d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:50.254509 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-a2da9111-149f-4474-8b32-4aa676155d54 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:50.255665 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a2da9111-149f-4474-8b32-4aa676155d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:50.264300 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810146a-810f-442d-9c83-3437c47f515b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:50.266887 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810146a-810f-442d-9c83-3437c47f515b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:50.267175 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810146a-810f-442d-9c83-3437c47f515b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:50.267451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810146a-810f-442d-9c83-3437c47f515b None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:50.268238 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b810146a-810f-442d-9c83-3437c47f515b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:50.268544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b810146a-810f-442d-9c83-3437c47f515b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:50.270103 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b810146a-810f-442d-9c83-3437c47f515b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:50.279849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 416/837] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:47:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:50.286155 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810146a-810f-442d-9c83-3437c47f515b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:50.348343 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 422/838] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:50 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:47:50.380485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:50.398168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810146a-810f-442d-9c83-3437c47f515b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:50.432249 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb01d82d-9e57-4d58-b31b-2b8531469819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:50.433143 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01d82d-9e57-4d58-b31b-2b8531469819 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:50.433584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01d82d-9e57-4d58-b31b-2b8531469819 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:50.434226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01d82d-9e57-4d58-b31b-2b8531469819 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:50.446957 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb01d82d-9e57-4d58-b31b-2b8531469819 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:50.448600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb01d82d-9e57-4d58-b31b-2b8531469819 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:50.463639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-bb01d82d-9e57-4d58-b31b-2b8531469819 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:50.463639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:50.463639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:50.463639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:50.463639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:50.463639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:50.463639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:50.463639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:50.463639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:50.463639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:50.463639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:50.463639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:50.463639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:50.463639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:50.463639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:50.463639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:50.464827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:50.464827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:50.464827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:50.464827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:50.464827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:50.464827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:50.464827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:50.464827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:50.464827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:50.464827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:50.464827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:50.464827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:50.464827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:50.464827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:50.464827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:50.464827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:50.464827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:50.465904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:50.465904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:50.465904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:50.467587 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 417/839] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:47:50.480362 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da9111-149f-4474-8b32-4aa676155d54 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:50.577562 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:50.584401 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da9111-149f-4474-8b32-4aa676155d54 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:50.619864 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad68c8d-7c38-47b9-a202-b2e08dbbb926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:50.620645 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad68c8d-7c38-47b9-a202-b2e08dbbb926 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:50.620929 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad68c8d-7c38-47b9-a202-b2e08dbbb926 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:50.621200 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad68c8d-7c38-47b9-a202-b2e08dbbb926 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:50.621849 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad68c8d-7c38-47b9-a202-b2e08dbbb926 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:50.622150 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad68c8d-7c38-47b9-a202-b2e08dbbb926 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:50.623515 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad68c8d-7c38-47b9-a202-b2e08dbbb926 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:50.640197 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 423/840] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:50.641716 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb01d82d-9e57-4d58-b31b-2b8531469819 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:50.655562 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad68c8d-7c38-47b9-a202-b2e08dbbb926 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:50.773297 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad68c8d-7c38-47b9-a202-b2e08dbbb926 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:50.779362 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb01d82d-9e57-4d58-b31b-2b8531469819 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:50.818215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e186db-231c-4040-b825-ce6c5a437d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:50.819780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e186db-231c-4040-b825-ce6c5a437d92 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:50.820482 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e186db-231c-4040-b825-ce6c5a437d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:50.820980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e186db-231c-4040-b825-ce6c5a437d92 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:50.821584 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac2f33-3cb4-4ec2-9e9c-21a9c82f87d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:50.822306 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac2f33-3cb4-4ec2-9e9c-21a9c82f87d4 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:50.822591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac2f33-3cb4-4ec2-9e9c-21a9c82f87d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:50.822864 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac2f33-3cb4-4ec2-9e9c-21a9c82f87d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:50.825128 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:50.831309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e186db-231c-4040-b825-ce6c5a437d92 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:50.833122 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e186db-231c-4040-b825-ce6c5a437d92 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:50.834825 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ac2f33-3cb4-4ec2-9e9c-21a9c82f87d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:47:50.841113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-75ac2f33-3cb4-4ec2-9e9c-21a9c82f87d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:50.841113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:50.841113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:50.841113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:50.841113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:50.841113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:50.841113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:50.841113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:50.841113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:50.841113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:50.841113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:50.841113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:50.841113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:47:50.841113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:47:50.841113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:47:50.841113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:47:50.842128 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:47:50.842128 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:47:50.842128 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:47:50.842128 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:47:50.842128 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:50.842128 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:50.842128 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:50.842128 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:50.842128 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:50.842128 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:50.842128 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:50.842128 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:50.842128 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:50.842128 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:50.842128 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:50.842128 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:50.842960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:50.842960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:50.842960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:50.842960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:50.842960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:47:50.842960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:47:50.842960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:47:50.842960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:47:50.842960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:47:50.842960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:47:50.842960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:47:50.842960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:50.847094 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 424/841] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Sep 9 20:47:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:50.847941 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac2f33-3cb4-4ec2-9e9c-21a9c82f87d4 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:50.848949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-60e186db-231c-4040-b825-ce6c5a437d92 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 20:47:50.848949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:50.848949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:50.848949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:50.848949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:50.848949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:50.848949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:50.848949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:50.848949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:50.848949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:50.848949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:50.848949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:50.848949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:47:50.848949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:47:50.848949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:47:50.848949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:47:50.849882 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:50.849882 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:50.849882 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:47:50.849882 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:50.849882 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:47:50.849882 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:50.849882 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:50.849882 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:50.849882 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:50.849882 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:50.849882 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:50.849882 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:50.849882 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:50.849882 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:50.849882 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:50.849882 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:50.849882 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:50.849882 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:50.850835 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:50.850835 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:50.850835 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:47:50.850835 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:47:50.850835 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:47:50.850835 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:47:50.850835 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:47:50.850835 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:47:50.850835 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 20:47:50.850835 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:50.857793 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 418/842] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:47:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:50.859247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e186db-231c-4040-b825-ce6c5a437d92 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:50.868643 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:50.965514 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac2f33-3cb4-4ec2-9e9c-21a9c82f87d4 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:50.971155 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e186db-231c-4040-b825-ce6c5a437d92 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:51.003546 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ef84a-2547-4263-aa1d-afd78d973538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:51.004186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ef84a-2547-4263-aa1d-afd78d973538 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:51.004470 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ef84a-2547-4263-aa1d-afd78d973538 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:51.004744 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ef84a-2547-4263-aa1d-afd78d973538 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:51.005465 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ef84a-2547-4263-aa1d-afd78d973538 None admin] RBAC: Adding query filter params (name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:51.005716 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ef84a-2547-4263-aa1d-afd78d973538 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:51.007135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ef84a-2547-4263-aa1d-afd78d973538 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:51.010105 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24548159-811d-43bb-931b-2cb86cf4886d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:51.010723 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24548159-811d-43bb-931b-2cb86cf4886d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:51.011003 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24548159-811d-43bb-931b-2cb86cf4886d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:51.011272 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24548159-811d-43bb-931b-2cb86cf4886d None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:51.012400 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24548159-811d-43bb-931b-2cb86cf4886d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:51.012400 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24548159-811d-43bb-931b-2cb86cf4886d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:51.013719 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24548159-811d-43bb-931b-2cb86cf4886d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:51.023407 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 425/843] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Sep 9 20:47:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:51.023407 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c369191-cd07-4df7-9953-5233d1e1cf17 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:51.023407 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c369191-cd07-4df7-9953-5233d1e1cf17 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:51.023407 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c369191-cd07-4df7-9953-5233d1e1cf17 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:51.023407 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 426/844] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:51.024906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c369191-cd07-4df7-9953-5233d1e1cf17 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:51.037007 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 419/845] 10.222.0.48 () {56 vars in 1135 bytes} [Fri Sep 9 20:47:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:51.038896 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c5fa9-dc46-4520-8936-48ecc450a30c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:51.039179 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c5fa9-dc46-4520-8936-48ecc450a30c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:51.039453 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c5fa9-dc46-4520-8936-48ecc450a30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:51.058160 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-897c5fa9-dc46-4520-8936-48ecc450a30c None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:51.059350 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-897c5fa9-dc46-4520-8936-48ecc450a30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:51.110156 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:51.112501 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:51.140095 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 420/846] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:47:51.140095 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c5fa9-dc46-4520-8936-48ecc450a30c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:51.142319 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c369191-cd07-4df7-9953-5233d1e1cf17 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:51.187981 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbb308-c8d9-40db-86c3-b006053404f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:51.188650 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbb308-c8d9-40db-86c3-b006053404f9 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:51.188930 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbb308-c8d9-40db-86c3-b006053404f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:51.189374 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbb308-c8d9-40db-86c3-b006053404f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:51.203245 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cbb308-c8d9-40db-86c3-b006053404f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:51.205222 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cbb308-c8d9-40db-86c3-b006053404f9 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:51.224435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-09cbb308-c8d9-40db-86c3-b006053404f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:51.224435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:51.224435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:51.224435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:51.224435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:51.224435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:51.224435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:51.224435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:51.224435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:51.224435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:51.224435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:51.224435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:51.224435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:51.224435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:51.224435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:51.224435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:51.225970 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:51.225970 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:51.225970 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:51.225970 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:51.225970 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:51.225970 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:51.225970 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:51.225970 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:51.225970 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:51.225970 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:51.225970 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:51.225970 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:51.225970 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:51.225970 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:51.225970 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:51.225970 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:51.225970 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:51.226869 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:51.226869 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:51.226869 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:51.226869 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 427/847] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:47:51.236106 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbb308-c8d9-40db-86c3-b006053404f9 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:51.253215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c5fa9-dc46-4520-8936-48ecc450a30c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:51.290048 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373527f-a35b-4188-ae54-fe25eb4132a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:51.291001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373527f-a35b-4188-ae54-fe25eb4132a8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:51.291555 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373527f-a35b-4188-ae54-fe25eb4132a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:51.292016 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373527f-a35b-4188-ae54-fe25eb4132a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:51.343931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbb308-c8d9-40db-86c3-b006053404f9 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:51.350917 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:51.371709 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373527f-a35b-4188-ae54-fe25eb4132a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:51.373912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373527f-a35b-4188-ae54-fe25eb4132a8 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:51.378287 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:51.387922 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae421eb8-8104-4895-913a-a38dd843baa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:51.388674 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae421eb8-8104-4895-913a-a38dd843baa0 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:51.388962 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae421eb8-8104-4895-913a-a38dd843baa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:51.389237 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae421eb8-8104-4895-913a-a38dd843baa0 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:51.389883 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae421eb8-8104-4895-913a-a38dd843baa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:51.390187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae421eb8-8104-4895-913a-a38dd843baa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:51.391710 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae421eb8-8104-4895-913a-a38dd843baa0 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:51.408840 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 428/848] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:47:51.410836 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae421eb8-8104-4895-913a-a38dd843baa0 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:51.481243 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 421/849] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:47:51 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:47:51.487374 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373527f-a35b-4188-ae54-fe25eb4132a8 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:51.517308 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae421eb8-8104-4895-913a-a38dd843baa0 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:51.551943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d229e9e-d67a-4873-bf0d-f67f17625920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:51.552538 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d229e9e-d67a-4873-bf0d-f67f17625920 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:51.552818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d229e9e-d67a-4873-bf0d-f67f17625920 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:51.553625 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d229e9e-d67a-4873-bf0d-f67f17625920 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:51.561057 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d229e9e-d67a-4873-bf0d-f67f17625920 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:51.562500 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d229e9e-d67a-4873-bf0d-f67f17625920 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:51.574725 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-3d229e9e-d67a-4873-bf0d-f67f17625920 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 20:47:51.574725 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:51.574725 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:51.574725 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:51.574725 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:51.574725 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:51.574725 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:51.574725 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:51.574725 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:51.574725 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:51.574725 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:51.574725 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:51.574725 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:47:51.574725 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:47:51.574725 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:47:51.574725 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:47:51.575687 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:51.575687 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:51.575687 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:47:51.575687 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:51.575687 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:47:51.575687 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:51.575687 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:51.575687 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:51.575687 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:51.575687 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:51.575687 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:51.575687 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:51.575687 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:51.575687 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:51.575687 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:51.575687 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:51.575687 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:51.575687 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:51.576674 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:51.576674 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:51.576674 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:47:51.576674 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:47:51.576674 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:47:51.576674 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:47:51.576674 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:47:51.576674 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:47:51.576674 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 20:47:51.576674 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:51.576674 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 429/850] 10.222.0.48 () {56 vars in 1117 bytes} [Fri Sep 9 20:47:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:51.576674 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d229e9e-d67a-4873-bf0d-f67f17625920 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:51.582823 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:51.598909 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373527f-a35b-4188-ae54-fe25eb4132a8 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:51.633414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:51.637347 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ff448-5be3-4303-ae0c-91e386e733e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:51.637980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ff448-5be3-4303-ae0c-91e386e733e1 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:51.638258 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ff448-5be3-4303-ae0c-91e386e733e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:51.638526 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ff448-5be3-4303-ae0c-91e386e733e1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:51.639110 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ff448-5be3-4303-ae0c-91e386e733e1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:51.640560 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ff448-5be3-4303-ae0c-91e386e733e1 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:51.649546 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-408ff448-5be3-4303-ae0c-91e386e733e1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 20:47:51.649546 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:51.649546 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:47:51.649546 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:47:51.649546 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:47:51.649546 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:47:51.649546 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:47:51.649546 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:47:51.649546 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 20:47:51.649546 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:51.649546 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:47:51.649546 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:51.649546 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:51.649546 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:51.649546 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:51.649546 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:51.649546 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:51.650666 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:51.650666 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:51.650666 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:51.650666 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:51.650666 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:51.650666 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:51.650666 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:47:51.650666 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:47:51.650666 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:47:51.650666 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:47:51.650666 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:51.650666 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:51.650666 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:51.650666 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:51.650666 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:51.650666 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:51.651522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:51.651522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:51.651522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:51.651522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:51.651522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:51.651522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:51.651522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:51.651522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:51.651522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:51.651522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:51.651522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:47:51.651522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:47:51.651522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 20:47:51.651522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:51.651522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 422/851] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Sep 9 20:47:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:51.652372 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ff448-5be3-4303-ae0c-91e386e733e1 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:51.689625 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d229e9e-d67a-4873-bf0d-f67f17625920 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:51.725468 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a5d05-bce9-4dc3-af13-31e0080b89f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:51.726076 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a5d05-bce9-4dc3-af13-31e0080b89f1 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:51.726354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a5d05-bce9-4dc3-af13-31e0080b89f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:51.726627 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a5d05-bce9-4dc3-af13-31e0080b89f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:51.737503 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a5d05-bce9-4dc3-af13-31e0080b89f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:51.738938 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a5d05-bce9-4dc3-af13-31e0080b89f1 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:51.753737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-0d5a5d05-bce9-4dc3-af13-31e0080b89f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:51.753737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:51.753737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:51.753737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:51.753737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:51.753737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:51.753737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:51.753737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:51.753737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:51.753737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:51.753737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:51.753737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:51.753737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:51.753737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:51.753737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:51.753737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:51.754740 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:51.754740 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:51.754740 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:51.754740 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:51.754740 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:51.754740 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:51.754740 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:51.754740 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:51.754740 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:51.754740 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:51.754740 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:51.754740 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:51.754740 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:51.754740 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:51.754740 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:51.754740 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:51.754740 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:51.760600 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:51.760600 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:51.760600 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:51.760783 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ff448-5be3-4303-ae0c-91e386e733e1 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:51.760878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 430/852] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:51.760878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a5d05-bce9-4dc3-af13-31e0080b89f1 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:51.794473 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619aca4-c9c9-44b9-b5dc-3db2679abdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:51.795179 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619aca4-c9c9-44b9-b5dc-3db2679abdc8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:51.795458 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619aca4-c9c9-44b9-b5dc-3db2679abdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:51.795731 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619aca4-c9c9-44b9-b5dc-3db2679abdc8 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:51.796480 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619aca4-c9c9-44b9-b5dc-3db2679abdc8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:51.796786 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619aca4-c9c9-44b9-b5dc-3db2679abdc8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:51.798208 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619aca4-c9c9-44b9-b5dc-3db2679abdc8 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:51.816725 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 423/853] 10.222.0.48 () {56 vars in 1149 bytes} [Fri Sep 9 20:47:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:51.817464 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619aca4-c9c9-44b9-b5dc-3db2679abdc8 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:51.825750 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:51.870001 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a5d05-bce9-4dc3-af13-31e0080b89f1 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:51.871802 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:51.909469 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c9991-e7ca-4294-bc2c-80cca520edc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:51.910088 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c9991-e7ca-4294-bc2c-80cca520edc9 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:51.910367 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c9991-e7ca-4294-bc2c-80cca520edc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:51.910636 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c9991-e7ca-4294-bc2c-80cca520edc9 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:51.911280 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c9991-e7ca-4294-bc2c-80cca520edc9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:51.911574 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c9991-e7ca-4294-bc2c-80cca520edc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:51.913565 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c9991-e7ca-4294-bc2c-80cca520edc9 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:51.923379 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 431/854] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Sep 9 20:47:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:51.923379 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c9991-e7ca-4294-bc2c-80cca520edc9 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:51.932525 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619aca4-c9c9-44b9-b5dc-3db2679abdc8 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:51.970012 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098cbc4-f75f-46e1-aef0-9886e532b683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:51.970622 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098cbc4-f75f-46e1-aef0-9886e532b683 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:51.970895 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098cbc4-f75f-46e1-aef0-9886e532b683 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:51.971165 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098cbc4-f75f-46e1-aef0-9886e532b683 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:51.971803 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098cbc4-f75f-46e1-aef0-9886e532b683 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:51.972127 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098cbc4-f75f-46e1-aef0-9886e532b683 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:51.973494 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098cbc4-f75f-46e1-aef0-9886e532b683 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:51.985745 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 424/855] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:51.986401 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098cbc4-f75f-46e1-aef0-9886e532b683 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:52.042178 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c9991-e7ca-4294-bc2c-80cca520edc9 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:52.059381 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:52.088751 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c236e7-5629-4a1d-9fd0-ac27241baf53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:52.088751 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c236e7-5629-4a1d-9fd0-ac27241baf53 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:52.088751 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c236e7-5629-4a1d-9fd0-ac27241baf53 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:52.088751 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c236e7-5629-4a1d-9fd0-ac27241baf53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:52.094203 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098cbc4-f75f-46e1-aef0-9886e532b683 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:52.110312 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:52.127724 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd1f03-3df8-4cc0-aee9-516007a757e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:52.128353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd1f03-3df8-4cc0-aee9-516007a757e3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:52.128636 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd1f03-3df8-4cc0-aee9-516007a757e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:52.128908 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd1f03-3df8-4cc0-aee9-516007a757e3 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:52.129578 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd1f03-3df8-4cc0-aee9-516007a757e3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:52.129880 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd1f03-3df8-4cc0-aee9-516007a757e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:52.131246 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd1f03-3df8-4cc0-aee9-516007a757e3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:52.135573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 425/856] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Sep 9 20:47:51 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:52.136433 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd1f03-3df8-4cc0-aee9-516007a757e3 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:52.144089 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c236e7-5629-4a1d-9fd0-ac27241baf53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:52.145562 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c236e7-5629-4a1d-9fd0-ac27241baf53 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:52.258996 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 432/857] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:47:51 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:47:52.258996 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d35f7c-e553-47c3-bff2-72fb3754d4d0 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:52.258996 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d35f7c-e553-47c3-bff2-72fb3754d4d0 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:52.258996 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d35f7c-e553-47c3-bff2-72fb3754d4d0 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:52.258996 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 433/858] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:52.261598 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d35f7c-e553-47c3-bff2-72fb3754d4d0 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:52.278382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd1f03-3df8-4cc0-aee9-516007a757e3 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:52.312366 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:52.319780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0719d929-9c45-4acd-85bf-92191c654b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:52.332590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719d929-9c45-4acd-85bf-92191c654b0e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:52.332590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719d929-9c45-4acd-85bf-92191c654b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:52.332590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719d929-9c45-4acd-85bf-92191c654b0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:52.332590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0719d929-9c45-4acd-85bf-92191c654b0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:52.332590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0719d929-9c45-4acd-85bf-92191c654b0e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:52.341346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-0719d929-9c45-4acd-85bf-92191c654b0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:52.341346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:52.341346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:52.341346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:52.341346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:52.341346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:52.341346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:52.341346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:52.341346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:52.341346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:52.341346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:52.341346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:52.341346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:47:52.341346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:47:52.341346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:47:52.342396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:47:52.342396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:52.342396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:52.342396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:52.342396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:52.342396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:52.342396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:52.342396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:52.342396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:52.342396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:52.342396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:52.342396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:52.342396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:52.342396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:52.342396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:52.342396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:52.342396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:52.343459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:47:52.343459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:47:52.343459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:47:52.343459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:47:52.343459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:47:52.343459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:47:52.343459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:52.343459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:52.343459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 426/859] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:47:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:52.344487 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a339e54-e250-42d7-b9d7-f0d50b7ce46b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:52.344764 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a339e54-e250-42d7-b9d7-f0d50b7ce46b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:52.345030 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a339e54-e250-42d7-b9d7-f0d50b7ce46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:52.351713 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:52.365351 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-2a339e54-e250-42d7-b9d7-f0d50b7ce46b None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:52.366782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2a339e54-e250-42d7-b9d7-f0d50b7ce46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:52.390272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d35f7c-e553-47c3-bff2-72fb3754d4d0 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:52.427394 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebdbdc7-7801-431b-97b6-0a7dabcf01e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:52.428198 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdbdc7-7801-431b-97b6-0a7dabcf01e9 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:52.428479 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdbdc7-7801-431b-97b6-0a7dabcf01e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:52.428756 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdbdc7-7801-431b-97b6-0a7dabcf01e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:52.435277 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 427/860] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:47:52.436558 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a339e54-e250-42d7-b9d7-f0d50b7ce46b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:52.450572 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebdbdc7-7801-431b-97b6-0a7dabcf01e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:52.452190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebdbdc7-7801-431b-97b6-0a7dabcf01e9 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:52.470650 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-2ebdbdc7-7801-431b-97b6-0a7dabcf01e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:52.470650 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:52.470650 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:52.470650 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:52.470650 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:52.470650 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:52.470650 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:52.470650 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:52.470650 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:52.470650 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:52.470650 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:52.470650 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:52.470650 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:52.470650 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:52.470650 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:52.470650 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:52.471647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:52.471647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:52.471647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:52.471647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:52.471647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:52.471647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:52.471647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:52.471647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:52.471647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:52.471647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:52.471647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:52.471647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:52.471647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:52.471647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:52.471647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:52.471647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:52.471647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:52.472916 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:52.472916 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:52.472916 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:52.476982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 434/861] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:52.478224 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebdbdc7-7801-431b-97b6-0a7dabcf01e9 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:52.552810 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a339e54-e250-42d7-b9d7-f0d50b7ce46b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:52.575691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:52.590532 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817032c4-b036-4f6d-b80c-2c19a6491805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:52.591143 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817032c4-b036-4f6d-b80c-2c19a6491805 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:52.591528 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817032c4-b036-4f6d-b80c-2c19a6491805 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:52.591753 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817032c4-b036-4f6d-b80c-2c19a6491805 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:52.592476 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817032c4-b036-4f6d-b80c-2c19a6491805 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:52.592783 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817032c4-b036-4f6d-b80c-2c19a6491805 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:52.594219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817032c4-b036-4f6d-b80c-2c19a6491805 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:52.606639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 428/862] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:47:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:52.607730 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebdbdc7-7801-431b-97b6-0a7dabcf01e9 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:52.614959 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817032c4-b036-4f6d-b80c-2c19a6491805 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:52.626250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:52.651164 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2d6da-5782-40cd-ae68-2aaee7bd7184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:52.651785 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2d6da-5782-40cd-ae68-2aaee7bd7184 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:52.652121 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2d6da-5782-40cd-ae68-2aaee7bd7184 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:52.652406 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2d6da-5782-40cd-ae68-2aaee7bd7184 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:52.654328 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2d6da-5782-40cd-ae68-2aaee7bd7184 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:52.654644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2d6da-5782-40cd-ae68-2aaee7bd7184 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:52.656167 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2d6da-5782-40cd-ae68-2aaee7bd7184 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:52.671703 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 435/863] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:52.676732 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2d6da-5782-40cd-ae68-2aaee7bd7184 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:52.732123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817032c4-b036-4f6d-b80c-2c19a6491805 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:52.771122 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d5ed9-5d49-484b-b194-3248b604b73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:52.771872 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d5ed9-5d49-484b-b194-3248b604b73b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:52.772277 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d5ed9-5d49-484b-b194-3248b604b73b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:52.772716 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d5ed9-5d49-484b-b194-3248b604b73b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:52.773538 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d5ed9-5d49-484b-b194-3248b604b73b None admin] RBAC: Adding query filter params (role.id=fcbf519fbc84458f913671a5a1d0914c, scope.project.id=91319aa782334fb9930770029b74e84a, user.id=7d28fa5d49d44120bd5eaec2d4e5fdbd) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:52.773955 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d5ed9-5d49-484b-b194-3248b604b73b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:52.775460 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d5ed9-5d49-484b-b194-3248b604b73b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:52.781741 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 429/864] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:47:52 2022] GET /identity/v3/role_assignments?user.id=7d28fa5d49d44120bd5eaec2d4e5fdbd&scope.project.id=91319aa782334fb9930770029b74e84a&role.id=fcbf519fbc84458f913671a5a1d0914c => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:52.782537 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d5ed9-5d49-484b-b194-3248b604b73b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:52.808318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2d6da-5782-40cd-ae68-2aaee7bd7184 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:52.835386 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:52.862096 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a162f1a-5897-4bd1-8e4e-b44912be417e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:52.863925 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a162f1a-5897-4bd1-8e4e-b44912be417e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:52.864382 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a162f1a-5897-4bd1-8e4e-b44912be417e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:52.864804 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a162f1a-5897-4bd1-8e4e-b44912be417e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:52.875020 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a162f1a-5897-4bd1-8e4e-b44912be417e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:52.877003 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a162f1a-5897-4bd1-8e4e-b44912be417e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:52.892943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-1a162f1a-5897-4bd1-8e4e-b44912be417e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:52.892943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:52.892943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:52.892943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:52.892943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:52.892943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:52.892943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:52.892943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:52.892943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:52.892943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:52.892943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:52.892943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:52.892943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:47:52.892943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:47:52.892943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:47:52.894093 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:47:52.894093 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:52.894093 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:52.894093 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:52.894093 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:52.894093 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:52.894093 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:52.894093 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:52.894093 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:52.894093 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:52.894093 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:52.894093 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:52.894093 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:52.894093 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:52.894093 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:52.894093 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:52.894093 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:52.895180 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:47:52.895180 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:47:52.895180 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:47:52.895180 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:47:52.895180 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:47:52.895180 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:47:52.895180 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:52.895180 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:52.901395 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 436/865] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:47:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:47:52.903099 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a162f1a-5897-4bd1-8e4e-b44912be417e None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:52.910876 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d5ed9-5d49-484b-b194-3248b604b73b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:52.954398 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:52.955260 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b66eee-6a50-4777-9583-21d6ce4b50c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:52.955939 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b66eee-6a50-4777-9583-21d6ce4b50c2 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:52.956278 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b66eee-6a50-4777-9583-21d6ce4b50c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:52.956555 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b66eee-6a50-4777-9583-21d6ce4b50c2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:52.957150 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b66eee-6a50-4777-9583-21d6ce4b50c2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:52.958539 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b66eee-6a50-4777-9583-21d6ce4b50c2 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:52.969889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-05b66eee-6a50-4777-9583-21d6ce4b50c2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 20:47:52.969889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:52.969889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:47:52.969889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:47:52.969889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:47:52.969889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:47:52.969889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:47:52.969889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:47:52.969889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 20:47:52.969889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:52.969889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:47:52.969889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:52.969889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:52.969889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:52.969889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:52.969889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:52.970890 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:52.970890 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:52.970890 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:52.970890 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:52.970890 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:52.970890 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:52.970890 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:52.970890 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:47:52.970890 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:47:52.970890 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:47:52.970890 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:47:52.970890 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:52.970890 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:52.970890 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:52.970890 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:52.970890 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:52.970890 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:52.971781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:52.971781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:52.971781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:52.971781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:52.971781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:52.971781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:52.971781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:52.971781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:52.971781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:52.971781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:52.971781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:47:52.971781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:47:52.971781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 20:47:52.971781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:52.971781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 430/866] 10.222.0.48 () {56 vars in 1132 bytes} [Fri Sep 9 20:47:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:52.977190 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b66eee-6a50-4777-9583-21d6ce4b50c2 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:53.032829 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a162f1a-5897-4bd1-8e4e-b44912be417e None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:53.076113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c579c832-9fbf-4c73-8237-f8fe44d602db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:53.076745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579c832-9fbf-4c73-8237-f8fe44d602db None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:53.077026 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579c832-9fbf-4c73-8237-f8fe44d602db None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:53.077301 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579c832-9fbf-4c73-8237-f8fe44d602db None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:53.078003 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c579c832-9fbf-4c73-8237-f8fe44d602db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:53.078310 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c579c832-9fbf-4c73-8237-f8fe44d602db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:53.079740 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c579c832-9fbf-4c73-8237-f8fe44d602db None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:53.098432 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 437/867] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:47:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:47:53.099874 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c579c832-9fbf-4c73-8237-f8fe44d602db None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:53.111929 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b66eee-6a50-4777-9583-21d6ce4b50c2 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:53.121873 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:53.148113 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e73e7b-bbd7-4706-9cca-2dc8ba2f49c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:53.148733 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e73e7b-bbd7-4706-9cca-2dc8ba2f49c3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:53.149011 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e73e7b-bbd7-4706-9cca-2dc8ba2f49c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:53.149323 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e73e7b-bbd7-4706-9cca-2dc8ba2f49c3 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:53.150017 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e73e7b-bbd7-4706-9cca-2dc8ba2f49c3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:53.150326 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e73e7b-bbd7-4706-9cca-2dc8ba2f49c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:53.151700 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e73e7b-bbd7-4706-9cca-2dc8ba2f49c3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:53.161005 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 431/868] 10.222.0.48 () {56 vars in 1126 bytes} [Fri Sep 9 20:47:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:53.162879 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e73e7b-bbd7-4706-9cca-2dc8ba2f49c3 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:53.220267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:53.224659 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c579c832-9fbf-4c73-8237-f8fe44d602db None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:53.268174 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bca219-3c29-4f91-84e4-6b8b7933cb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:53.268789 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bca219-3c29-4f91-84e4-6b8b7933cb34 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:53.269065 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bca219-3c29-4f91-84e4-6b8b7933cb34 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:53.269336 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bca219-3c29-4f91-84e4-6b8b7933cb34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:53.270064 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bca219-3c29-4f91-84e4-6b8b7933cb34 None admin] RBAC: Adding query filter params (role.id=1e000597af164d54a40df2a39c729820, scope.project.id=91319aa782334fb9930770029b74e84a, user.id=8c21ea7d3cbb494886e60a4bd4f39145) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:53.270370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bca219-3c29-4f91-84e4-6b8b7933cb34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:53.272014 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bca219-3c29-4f91-84e4-6b8b7933cb34 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:53.284099 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 438/869] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:47:53 2022] GET /identity/v3/role_assignments?user.id=8c21ea7d3cbb494886e60a4bd4f39145&scope.project.id=91319aa782334fb9930770029b74e84a&role.id=1e000597af164d54a40df2a39c729820 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:53.303674 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e73e7b-bbd7-4706-9cca-2dc8ba2f49c3 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:53.342336 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a959b-686c-405d-9f25-00de62a18811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:53.343014 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a959b-686c-405d-9f25-00de62a18811 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:53.343296 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a959b-686c-405d-9f25-00de62a18811 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:53.343566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a959b-686c-405d-9f25-00de62a18811 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:53.344252 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35a959b-686c-405d-9f25-00de62a18811 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:53.344559 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35a959b-686c-405d-9f25-00de62a18811 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:53.346093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35a959b-686c-405d-9f25-00de62a18811 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:53.352446 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 432/870] 10.222.0.48 () {56 vars in 1126 bytes} [Fri Sep 9 20:47:53 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:54.338194 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbe9cc-978b-4dba-b656-27157e90c853 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:54.338936 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbe9cc-978b-4dba-b656-27157e90c853 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:54.338936 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbe9cc-978b-4dba-b656-27157e90c853 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:54.340246 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 439/871] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:54.345830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951cc934-618e-48e2-aa86-c7a23c916f0a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:54.346253 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951cc934-618e-48e2-aa86-c7a23c916f0a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:54.346425 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951cc934-618e-48e2-aa86-c7a23c916f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:54.364823 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-951cc934-618e-48e2-aa86-c7a23c916f0a None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:54.366537 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-951cc934-618e-48e2-aa86-c7a23c916f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:54.396510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:54.437268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 433/872] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:47:54.702291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abbe9cc-978b-4dba-b656-27157e90c853 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:54.814263 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abbe9cc-978b-4dba-b656-27157e90c853 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:54.848444 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c382c4d-b252-4975-b0c1-ec91c86e4b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:54.849161 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c382c4d-b252-4975-b0c1-ec91c86e4b3f None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:54.849542 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c382c4d-b252-4975-b0c1-ec91c86e4b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:54.850343 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c382c4d-b252-4975-b0c1-ec91c86e4b3f None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:54.851045 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c382c4d-b252-4975-b0c1-ec91c86e4b3f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:54.852416 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c382c4d-b252-4975-b0c1-ec91c86e4b3f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:54.924430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 440/873] 10.222.0.48 () {60 vars in 1141 bytes} [Fri Sep 9 20:47:54 2022] POST /identity/v3/services => generated 251 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:47:55.138738 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d101ef-be70-4876-94f2-a784c6ad7b3a None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:55.139431 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d101ef-be70-4876-94f2-a784c6ad7b3a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:55.140011 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d101ef-be70-4876-94f2-a784c6ad7b3a None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:55.142985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 434/874] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:55.149864 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10867a-7ea0-49df-b679-6be484068cc4 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:55.150449 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10867a-7ea0-49df-b679-6be484068cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:55.150919 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10867a-7ea0-49df-b679-6be484068cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:55.169736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-3b10867a-7ea0-49df-b679-6be484068cc4 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:55.171268 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3b10867a-7ea0-49df-b679-6be484068cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:55.188683 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:55.248026 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 441/875] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:55 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:47:55.522280 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d101ef-be70-4876-94f2-a784c6ad7b3a None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:55.560660 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6995d1-be3d-47eb-a1ca-f3151fcb1b3f None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:55.561471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6995d1-be3d-47eb-a1ca-f3151fcb1b3f None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:55.561471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6995d1-be3d-47eb-a1ca-f3151fcb1b3f None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:55.563232 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 442/876] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:55.569514 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286c41b-91a8-4684-bfe7-95ad5bca2999 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:55.569833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286c41b-91a8-4684-bfe7-95ad5bca2999 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:55.570060 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286c41b-91a8-4684-bfe7-95ad5bca2999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:55.590252 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-f286c41b-91a8-4684-bfe7-95ad5bca2999 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:55.593872 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f286c41b-91a8-4684-bfe7-95ad5bca2999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:55.637696 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d101ef-be70-4876-94f2-a784c6ad7b3a None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:55.683445 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 443/877] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:55 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:47:55.686230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987ccff-2f4f-4e44-8fdd-a2b84572e860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:55.687169 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987ccff-2f4f-4e44-8fdd-a2b84572e860 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:55.687707 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987ccff-2f4f-4e44-8fdd-a2b84572e860 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:55.688276 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987ccff-2f4f-4e44-8fdd-a2b84572e860 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:55.689114 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6987ccff-2f4f-4e44-8fdd-a2b84572e860 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:55.689883 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fa033-22f9-4aca-8c48-679745a47a15 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:55.690185 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fa033-22f9-4aca-8c48-679745a47a15 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:55.690442 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fa033-22f9-4aca-8c48-679745a47a15 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:55.691120 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6987ccff-2f4f-4e44-8fdd-a2b84572e860 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:55.693166 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 444/878] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:55.697908 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd469eba-47e3-4edc-bafa-9d86b1b1bd50 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:55.698195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd469eba-47e3-4edc-bafa-9d86b1b1bd50 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:55.698471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd469eba-47e3-4edc-bafa-9d86b1b1bd50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:55.702468 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-6987ccff-2f4f-4e44-8fdd-a2b84572e860 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 20:47:55.702468 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:55.702468 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:47:55.702468 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:47:55.702468 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:47:55.702468 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:47:55.702468 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:47:55.702468 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:47:55.702468 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 20:47:55.702468 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:55.702468 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:47:55.702468 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:55.702468 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:55.702468 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:55.702468 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:55.702468 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:55.702468 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:55.703451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:55.703451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:55.703451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:55.703451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:55.703451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:55.703451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:55.703451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:47:55.703451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:47:55.703451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:47:55.703451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:47:55.703451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:55.703451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:55.703451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:55.703451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:55.703451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:55.703451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:55.704306 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:55.704306 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:55.704306 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:55.704306 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:55.704306 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:55.704306 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:55.704306 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:55.704306 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:55.704306 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:55.704306 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:55.704306 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:47:55.704306 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:47:55.704306 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 20:47:55.704306 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:55.705620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 435/879] 10.222.0.48 () {56 vars in 1126 bytes} [Fri Sep 9 20:47:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:55.716184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987ccff-2f4f-4e44-8fdd-a2b84572e860 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:55.720628 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-cd469eba-47e3-4edc-bafa-9d86b1b1bd50 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:55.721820 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-cd469eba-47e3-4edc-bafa-9d86b1b1bd50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:55.748438 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:55.807582 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 445/880] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:55 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:47:55.824685 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987ccff-2f4f-4e44-8fdd-a2b84572e860 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:55.861940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583464d-c851-4d55-a575-f0a49ad06e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:55.862558 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583464d-c851-4d55-a575-f0a49ad06e18 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:55.862834 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583464d-c851-4d55-a575-f0a49ad06e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:55.863104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583464d-c851-4d55-a575-f0a49ad06e18 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:55.863737 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0583464d-c851-4d55-a575-f0a49ad06e18 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:55.864037 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0583464d-c851-4d55-a575-f0a49ad06e18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:55.865985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0583464d-c851-4d55-a575-f0a49ad06e18 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:55.871309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 436/881] 10.222.0.48 () {56 vars in 1122 bytes} [Fri Sep 9 20:47:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:55.876731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd469eba-47e3-4edc-bafa-9d86b1b1bd50 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:55.888406 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:55.980122 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd469eba-47e3-4edc-bafa-9d86b1b1bd50 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:55.980311 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583464d-c851-4d55-a575-f0a49ad06e18 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:56.017087 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239eca0b-f6a1-4bd4-a42b-4f2e7543eba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:56.017700 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239eca0b-f6a1-4bd4-a42b-4f2e7543eba0 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:56.018025 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239eca0b-f6a1-4bd4-a42b-4f2e7543eba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:56.018298 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239eca0b-f6a1-4bd4-a42b-4f2e7543eba0 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:56.018925 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239eca0b-f6a1-4bd4-a42b-4f2e7543eba0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:56.019385 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239eca0b-f6a1-4bd4-a42b-4f2e7543eba0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:56.020767 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239eca0b-f6a1-4bd4-a42b-4f2e7543eba0 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:56.026267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 446/882] 10.222.0.48 () {56 vars in 1122 bytes} [Fri Sep 9 20:47:55 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:56.092293 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239eca0b-f6a1-4bd4-a42b-4f2e7543eba0 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:56.098225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:56.106640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583464d-c851-4d55-a575-f0a49ad06e18 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:56.153146 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14085072-d60f-4d14-b2dc-acfc747ffabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:56.153797 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14085072-d60f-4d14-b2dc-acfc747ffabc None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:56.154077 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14085072-d60f-4d14-b2dc-acfc747ffabc None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:56.154502 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14085072-d60f-4d14-b2dc-acfc747ffabc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:56.171014 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14085072-d60f-4d14-b2dc-acfc747ffabc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:47:56.179490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-14085072-d60f-4d14-b2dc-acfc747ffabc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 20:47:56.179490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:56.179490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:56.179490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:56.179490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:56.179490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:56.179490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:56.179490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:56.179490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:56.179490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:56.179490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:56.179490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:56.179490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:47:56.179490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:47:56.179490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:47:56.179490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:47:56.180780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:47:56.180780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:47:56.180780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:47:56.180780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:47:56.180780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:56.180780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:56.180780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:56.180780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:56.180780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:56.180780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:56.180780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:56.180780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:56.180780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:56.180780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:56.180780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:56.180780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:56.181621 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:56.181621 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:56.181621 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:56.181621 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:56.181621 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:47:56.181621 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:47:56.181621 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:47:56.181621 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:47:56.181621 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:47:56.181621 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:47:56.181621 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 20:47:56.181621 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:56.185864 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 437/883] 10.222.0.48 () {56 vars in 1120 bytes} [Fri Sep 9 20:47:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:47:56.187500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14085072-d60f-4d14-b2dc-acfc747ffabc None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:56.187610 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:56.217944 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239eca0b-f6a1-4bd4-a42b-4f2e7543eba0 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:56.257821 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c3a86-a061-454c-a3ab-9d0208362c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:56.258543 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c3a86-a061-454c-a3ab-9d0208362c85 None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:56.258822 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c3a86-a061-454c-a3ab-9d0208362c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:56.259192 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c3a86-a061-454c-a3ab-9d0208362c85 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:56.259747 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c3a86-a061-454c-a3ab-9d0208362c85 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:56.261398 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c3a86-a061-454c-a3ab-9d0208362c85 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:56.305878 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14085072-d60f-4d14-b2dc-acfc747ffabc None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:56.342019 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b47fde-f776-4afd-9a23-0cc5bac4bd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:56.343022 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b47fde-f776-4afd-9a23-0cc5bac4bd6f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:56.343504 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b47fde-f776-4afd-9a23-0cc5bac4bd6f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:56.344019 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b47fde-f776-4afd-9a23-0cc5bac4bd6f None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:56.344924 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b47fde-f776-4afd-9a23-0cc5bac4bd6f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:56.345478 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b47fde-f776-4afd-9a23-0cc5bac4bd6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:56.347149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b47fde-f776-4afd-9a23-0cc5bac4bd6f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:56.357716 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 438/884] 10.222.0.48 () {56 vars in 1115 bytes} [Fri Sep 9 20:47:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:56.365025 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b47fde-f776-4afd-9a23-0cc5bac4bd6f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:56.370413 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 447/885] 10.222.0.48 () {60 vars in 1141 bytes} [Fri Sep 9 20:47:56 2022] POST /identity/v3/services => generated 269 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:47:56.390493 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:56.490240 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b47fde-f776-4afd-9a23-0cc5bac4bd6f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:56.535502 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa5c49-becf-447b-95bc-45ec280d479b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:56.536198 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa5c49-becf-447b-95bc-45ec280d479b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:56.536436 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa5c49-becf-447b-95bc-45ec280d479b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:56.536714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa5c49-becf-447b-95bc-45ec280d479b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:56.549863 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa5c49-becf-447b-95bc-45ec280d479b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:56.551351 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa5c49-becf-447b-95bc-45ec280d479b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:56.567444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-93aa5c49-becf-447b-95bc-45ec280d479b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:56.567444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:56.567444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:56.567444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:56.567444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:56.567444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:56.567444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:56.567444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:56.567444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:56.567444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:56.567444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:56.567444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:56.567444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:56.567444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:56.567444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:56.567444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:56.568538 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:56.568538 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:56.568538 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:56.568538 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:56.568538 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:56.568538 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:56.568538 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:56.568538 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:56.568538 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:56.568538 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:56.568538 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:56.568538 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:56.568538 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:56.568538 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:56.568538 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:56.568538 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:56.568538 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:56.570411 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:56.570411 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:56.570411 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:56.573326 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 439/886] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:47:56.574685 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c3a86-a061-454c-a3ab-9d0208362c85 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:56.682428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c3a86-a061-454c-a3ab-9d0208362c85 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:56.712948 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:56.718633 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec7b95-0be3-4f79-abdf-1144959ed378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:56.719253 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec7b95-0be3-4f79-abdf-1144959ed378 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:56.719588 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec7b95-0be3-4f79-abdf-1144959ed378 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:56.719868 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec7b95-0be3-4f79-abdf-1144959ed378 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:56.720595 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec7b95-0be3-4f79-abdf-1144959ed378 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:56.720900 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec7b95-0be3-4f79-abdf-1144959ed378 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:56.722276 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec7b95-0be3-4f79-abdf-1144959ed378 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:56.739657 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 448/887] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:56.741113 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa5c49-becf-447b-95bc-45ec280d479b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:56.817244 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:56.858792 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa5c49-becf-447b-95bc-45ec280d479b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:56.895347 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e594545-5e62-4ec4-9c79-719c195e0526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:56.896054 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e594545-5e62-4ec4-9c79-719c195e0526 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:56.896356 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e594545-5e62-4ec4-9c79-719c195e0526 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:56.897221 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e594545-5e62-4ec4-9c79-719c195e0526 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:56.905285 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e594545-5e62-4ec4-9c79-719c195e0526 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:56.906742 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e594545-5e62-4ec4-9c79-719c195e0526 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:56.918157 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-4e594545-5e62-4ec4-9c79-719c195e0526 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 20:47:56.918157 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:56.918157 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:56.918157 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:56.918157 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:56.918157 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:56.918157 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:56.918157 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:56.918157 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:56.918157 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:56.918157 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:56.918157 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:56.918157 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:47:56.918157 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:47:56.918157 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:47:56.918157 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:47:56.919135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:56.919135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:56.919135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:47:56.919135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:56.919135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:47:56.919135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:47:56.919135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:56.919135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:56.919135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:56.919135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:56.919135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:56.919135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:56.919135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:56.919135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:56.919135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:56.919135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:56.919135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:56.919135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:56.920099 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:56.920099 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:56.920099 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:47:56.920099 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:47:56.920099 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:47:56.920099 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:47:56.920099 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:47:56.920099 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:47:56.920099 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 20:47:56.920099 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:56.923406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 440/888] 10.222.0.48 () {56 vars in 1117 bytes} [Fri Sep 9 20:47:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:56.925425 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec7b95-0be3-4f79-abdf-1144959ed378 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:57.036887 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec7b95-0be3-4f79-abdf-1144959ed378 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:57.076718 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0592d58e-e9d3-4fac-b225-c4fec3d3ed18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:57.077679 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592d58e-e9d3-4fac-b225-c4fec3d3ed18 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:57.078218 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592d58e-e9d3-4fac-b225-c4fec3d3ed18 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:57.078674 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592d58e-e9d3-4fac-b225-c4fec3d3ed18 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:57.079603 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0592d58e-e9d3-4fac-b225-c4fec3d3ed18 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:57.080111 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0592d58e-e9d3-4fac-b225-c4fec3d3ed18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:57.081867 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0592d58e-e9d3-4fac-b225-c4fec3d3ed18 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:57.101437 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 449/889] 10.222.0.48 () {56 vars in 1149 bytes} [Fri Sep 9 20:47:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:57.108123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e594545-5e62-4ec4-9c79-719c195e0526 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:57.170800 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c1178-1a07-4b02-90a1-d34956aaf096 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:57.171465 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c1178-1a07-4b02-90a1-d34956aaf096 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:57.171947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c1178-1a07-4b02-90a1-d34956aaf096 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:57.173776 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 450/890] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:57.178778 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45263b66-58f3-4dec-8b93-92d0d3f32d9c None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:57.179061 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45263b66-58f3-4dec-8b93-92d0d3f32d9c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:57.179335 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45263b66-58f3-4dec-8b93-92d0d3f32d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:57.199988 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-45263b66-58f3-4dec-8b93-92d0d3f32d9c None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:57.201405 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-45263b66-58f3-4dec-8b93-92d0d3f32d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:57.214520 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e594545-5e62-4ec4-9c79-719c195e0526 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:57.239179 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:57.252218 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e220a-fda6-49d0-831f-e49ed4d31237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:57.253964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e220a-fda6-49d0-831f-e49ed4d31237 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:57.253964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e220a-fda6-49d0-831f-e49ed4d31237 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:57.253964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e220a-fda6-49d0-831f-e49ed4d31237 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:57.254730 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:57.288173 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 451/891] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:57 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:47:57.331570 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e220a-fda6-49d0-831f-e49ed4d31237 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:57.333109 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e220a-fda6-49d0-831f-e49ed4d31237 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:57.444226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 441/892] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:47:57 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:47:57.445819 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45263b66-58f3-4dec-8b93-92d0d3f32d9c None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:57.550150 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45263b66-58f3-4dec-8b93-92d0d3f32d9c None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:57.571824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e220a-fda6-49d0-831f-e49ed4d31237 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:57.590761 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc55db5-4074-46cb-87ff-c1da8af78efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:57.597319 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc55db5-4074-46cb-87ff-c1da8af78efe None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:57.597319 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc55db5-4074-46cb-87ff-c1da8af78efe None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:57.597319 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc55db5-4074-46cb-87ff-c1da8af78efe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:57.601460 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:57.605003 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc55db5-4074-46cb-87ff-c1da8af78efe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:57.606990 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc55db5-4074-46cb-87ff-c1da8af78efe None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:57.625499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-efc55db5-4074-46cb-87ff-c1da8af78efe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:57.625499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:57.625499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:57.625499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:57.625499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:57.625499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:57.625499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:57.625499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:57.625499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:57.625499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:57.625499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:57.625499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:57.625499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:47:57.625499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:47:57.625499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:47:57.625499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:47:57.626453 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:57.626453 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:57.626453 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:57.626453 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:57.626453 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:57.626453 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:57.626453 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:57.626453 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:57.626453 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:57.626453 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:57.626453 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:57.626453 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:57.626453 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:57.626453 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:57.626453 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:57.626453 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:57.626453 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:47:57.627353 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:47:57.627353 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:47:57.627353 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:57.632094 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 452/893] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:47:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:57.633688 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc55db5-4074-46cb-87ff-c1da8af78efe None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:57.672218 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:57.700763 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e220a-fda6-49d0-831f-e49ed4d31237 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:57.745189 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c7c35-772b-4092-ab52-501699832653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:57.746617 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c7c35-772b-4092-ab52-501699832653 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:57.747133 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c7c35-772b-4092-ab52-501699832653 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:57.747871 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c7c35-772b-4092-ab52-501699832653 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:57.748849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c7c35-772b-4092-ab52-501699832653 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:57.751089 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c7c35-772b-4092-ab52-501699832653 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:57.755888 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc55db5-4074-46cb-87ff-c1da8af78efe None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:57.762722 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-af7c7c35-772b-4092-ab52-501699832653 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 20:47:57.762722 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:57.762722 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:47:57.762722 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:47:57.762722 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:47:57.762722 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:47:57.762722 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:47:57.762722 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:47:57.762722 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 20:47:57.762722 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:57.762722 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:47:57.762722 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:57.762722 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:57.762722 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:57.762722 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:57.762722 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:57.762722 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:57.763712 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:57.763712 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:57.763712 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:57.763712 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:57.763712 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:57.763712 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:57.763712 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:47:57.763712 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:47:57.763712 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:47:57.763712 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:47:57.763712 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:57.763712 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:57.763712 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:57.763712 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:57.763712 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:57.763712 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:57.764601 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:57.764601 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:57.764601 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:57.764601 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:57.764601 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:57.764601 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:57.764601 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:57.764601 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:57.764601 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:57.764601 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:57.764601 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:47:57.764601 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:47:57.764601 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 20:47:57.764601 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:57.766216 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 442/894] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Sep 9 20:47:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:47:57.777506 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c7c35-772b-4092-ab52-501699832653 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:57.800129 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b9fa0c-e527-42f8-9da4-d042f2c28afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:57.801259 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9fa0c-e527-42f8-9da4-d042f2c28afb None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:57.801823 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9fa0c-e527-42f8-9da4-d042f2c28afb None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:57.802298 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9fa0c-e527-42f8-9da4-d042f2c28afb None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:57.803702 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b9fa0c-e527-42f8-9da4-d042f2c28afb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:57.804268 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b9fa0c-e527-42f8-9da4-d042f2c28afb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:57.805963 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b9fa0c-e527-42f8-9da4-d042f2c28afb None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:57.823663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 453/895] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:47:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:57.824932 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b9fa0c-e527-42f8-9da4-d042f2c28afb None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:57.889352 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:57.908785 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c7c35-772b-4092-ab52-501699832653 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:57.934464 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:57.945534 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b9fa0c-e527-42f8-9da4-d042f2c28afb None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:57.958065 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ab19a-9fc7-4ba3-b2fd-0d1c216b0cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:57.958992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ab19a-9fc7-4ba3-b2fd-0d1c216b0cac None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:57.959471 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ab19a-9fc7-4ba3-b2fd-0d1c216b0cac None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:57.960103 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ab19a-9fc7-4ba3-b2fd-0d1c216b0cac None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:57.960983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468ab19a-9fc7-4ba3-b2fd-0d1c216b0cac None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:57.961482 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468ab19a-9fc7-4ba3-b2fd-0d1c216b0cac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:57.963079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468ab19a-9fc7-4ba3-b2fd-0d1c216b0cac None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:57.970096 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 443/896] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Sep 9 20:47:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:57.976676 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ab19a-9fc7-4ba3-b2fd-0d1c216b0cac None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:57.991285 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84240ebc-9a59-4b6b-b993-2f5dad1888f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:57.991922 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84240ebc-9a59-4b6b-b993-2f5dad1888f4 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:57.992237 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84240ebc-9a59-4b6b-b993-2f5dad1888f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:57.992514 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84240ebc-9a59-4b6b-b993-2f5dad1888f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:58.000498 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84240ebc-9a59-4b6b-b993-2f5dad1888f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:58.002357 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84240ebc-9a59-4b6b-b993-2f5dad1888f4 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:58.014539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-84240ebc-9a59-4b6b-b993-2f5dad1888f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:58.014539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:58.014539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:58.014539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:58.014539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:58.014539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:58.014539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:58.014539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:58.014539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:58.014539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:58.014539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:58.014539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:58.014539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:47:58.014539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:47:58.014539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:47:58.015414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:47:58.015414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:58.015414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:58.015414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:58.015414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:58.015414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:58.015414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:58.015414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:58.015414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:58.015414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:58.015414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:58.015414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:58.015414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:58.015414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:58.015414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:58.015414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:58.015414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:58.016454 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:47:58.016454 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:47:58.016454 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:47:58.016454 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:47:58.016454 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:47:58.016454 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:47:58.016454 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:47:58.016454 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:47:58.018487 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 454/897] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:47:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:58.024593 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84240ebc-9a59-4b6b-b993-2f5dad1888f4 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:58.115573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ab19a-9fc7-4ba3-b2fd-0d1c216b0cac None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:58.143971 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84240ebc-9a59-4b6b-b993-2f5dad1888f4 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:58.159170 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156bc34-b45a-415d-8547-4919710de15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:58.160247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156bc34-b45a-415d-8547-4919710de15a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:58.161078 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156bc34-b45a-415d-8547-4919710de15a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:58.161851 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156bc34-b45a-415d-8547-4919710de15a None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:58.163254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9156bc34-b45a-415d-8547-4919710de15a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:58.163945 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9156bc34-b45a-415d-8547-4919710de15a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:58.165876 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9156bc34-b45a-415d-8547-4919710de15a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:58.176987 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 444/898] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Sep 9 20:47:57 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:47:58.178270 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156bc34-b45a-415d-8547-4919710de15a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:58.189680 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23f7cb-86ca-47bd-bb71-45547a123373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:58.190293 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23f7cb-86ca-47bd-bb71-45547a123373 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:58.190570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23f7cb-86ca-47bd-bb71-45547a123373 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:58.190841 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23f7cb-86ca-47bd-bb71-45547a123373 None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:58.191780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23f7cb-86ca-47bd-bb71-45547a123373 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:58.191985 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23f7cb-86ca-47bd-bb71-45547a123373 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:58.194673 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23f7cb-86ca-47bd-bb71-45547a123373 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:58.203115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:58.212265 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 455/899] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:47:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:58.218924 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23f7cb-86ca-47bd-bb71-45547a123373 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:58.243405 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:58.313115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156bc34-b45a-415d-8547-4919710de15a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:58.338038 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23f7cb-86ca-47bd-bb71-45547a123373 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:58.351094 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d2c3d-ae35-4924-9183-5be3d321a3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:58.351094 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d2c3d-ae35-4924-9183-5be3d321a3a1 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:58.351094 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d2c3d-ae35-4924-9183-5be3d321a3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:58.351430 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d2c3d-ae35-4924-9183-5be3d321a3a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:58.351962 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d2c3d-ae35-4924-9183-5be3d321a3a1 None admin] RBAC: Adding query filter params (interface=public, service_id=1b71b0fe2dc04b4da69b4d916cab1415, region_id=RegionOne) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:58.352303 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d2c3d-ae35-4924-9183-5be3d321a3a1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:58.353665 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d2c3d-ae35-4924-9183-5be3d321a3a1 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:58.366479 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 445/900] 10.222.0.48 () {56 vars in 1251 bytes} [Fri Sep 9 20:47:58 2022] GET /identity/v3/endpoints?service_id=1b71b0fe2dc04b4da69b4d916cab1415&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:58.366479 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14524c-43e8-4361-b5b5-8fa984f22738 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:58.366479 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14524c-43e8-4361-b5b5-8fa984f22738 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:58.366479 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14524c-43e8-4361-b5b5-8fa984f22738 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:58.366479 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 446/901] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:58.369307 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14524c-43e8-4361-b5b5-8fa984f22738 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:58.378322 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db57a73-05fc-4493-a982-6caa30bcb98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:58.378979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db57a73-05fc-4493-a982-6caa30bcb98c None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:58.379257 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db57a73-05fc-4493-a982-6caa30bcb98c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:58.379587 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db57a73-05fc-4493-a982-6caa30bcb98c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:58.380367 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db57a73-05fc-4493-a982-6caa30bcb98c None admin] RBAC: Adding query filter params (role.id=63740f83debd49b6a81f640fe2a07ce2, scope.project.id=91319aa782334fb9930770029b74e84a, user.id=8c21ea7d3cbb494886e60a4bd4f39145) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:58.380679 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db57a73-05fc-4493-a982-6caa30bcb98c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:58.382647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db57a73-05fc-4493-a982-6caa30bcb98c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:58.388667 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 456/902] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:47:58 2022] GET /identity/v3/role_assignments?user.id=8c21ea7d3cbb494886e60a4bd4f39145&scope.project.id=91319aa782334fb9930770029b74e84a&role.id=63740f83debd49b6a81f640fe2a07ce2 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:58.389965 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c3b8c-fa22-4675-8856-98475d4f1d72 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:58.390271 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c3b8c-fa22-4675-8856-98475d4f1d72 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:58.390561 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c3b8c-fa22-4675-8856-98475d4f1d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:58.416443 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-d99c3b8c-fa22-4675-8856-98475d4f1d72 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:58.417723 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d99c3b8c-fa22-4675-8856-98475d4f1d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:58.473386 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:58.497585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14524c-43e8-4361-b5b5-8fa984f22738 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:58.509017 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 457/903] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:58 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:47:58.549241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbef18-f611-4e23-951b-4249eebd0b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:58.550359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbef18-f611-4e23-951b-4249eebd0b6c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:58.550809 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbef18-f611-4e23-951b-4249eebd0b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:58.551109 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbef18-f611-4e23-951b-4249eebd0b6c None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:58.551717 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbbef18-f611-4e23-951b-4249eebd0b6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:58.553296 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbbef18-f611-4e23-951b-4249eebd0b6c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:58.561786 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 447/904] 10.222.0.48 () {56 vars in 1087 bytes} [Fri Sep 9 20:47:58 2022] GET /identity/v3/services => generated 1268 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:47:58.787869 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c3b8c-fa22-4675-8856-98475d4f1d72 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:58.796846 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:58.821851 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc288b-762b-48a6-846a-1eb802b735d2 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:58.822357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc288b-762b-48a6-846a-1eb802b735d2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:58.822434 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc288b-762b-48a6-846a-1eb802b735d2 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:58.823873 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 448/905] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:47:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:47:58.829865 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db7815-808a-4bfa-9538-723266f9f710 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:58.829865 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db7815-808a-4bfa-9538-723266f9f710 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:58.829865 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db7815-808a-4bfa-9538-723266f9f710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:58.849958 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e4db7815-808a-4bfa-9538-723266f9f710 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:47:58.851898 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e4db7815-808a-4bfa-9538-723266f9f710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:47:58.901821 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c3b8c-fa22-4675-8856-98475d4f1d72 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:58.940653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819eca62-40d1-4804-b5bc-573f8bd5c15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:58.941423 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819eca62-40d1-4804-b5bc-573f8bd5c15f None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:58.941844 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819eca62-40d1-4804-b5bc-573f8bd5c15f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:58.942218 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819eca62-40d1-4804-b5bc-573f8bd5c15f None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:58.943033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819eca62-40d1-4804-b5bc-573f8bd5c15f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:58.945652 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819eca62-40d1-4804-b5bc-573f8bd5c15f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:58.947405 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 449/906] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:47:58 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:47:59.040810 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 458/907] 10.222.0.48 () {60 vars in 1141 bytes} [Fri Sep 9 20:47:58 2022] POST /identity/v3/services => generated 241 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:47:59.214526 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db7815-808a-4bfa-9538-723266f9f710 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:59.296725 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:59.324917 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db7815-808a-4bfa-9538-723266f9f710 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:59.366811 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2a041-1683-49d1-83f6-8885815bed41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:59.367444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2a041-1683-49d1-83f6-8885815bed41 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:59.367719 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2a041-1683-49d1-83f6-8885815bed41 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:59.367990 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2a041-1683-49d1-83f6-8885815bed41 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:59.368621 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec2a041-1683-49d1-83f6-8885815bed41 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:59.370000 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec2a041-1683-49d1-83f6-8885815bed41 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:59.378894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-1ec2a041-1683-49d1-83f6-8885815bed41 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 20:47:59.378894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:59.378894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:47:59.378894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:47:59.378894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:47:59.378894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:47:59.378894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:47:59.378894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:47:59.378894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 20:47:59.378894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:59.378894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:47:59.378894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:59.378894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:47:59.378894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:47:59.378894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:47:59.378894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:47:59.379824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:47:59.379824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:47:59.379824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:47:59.379824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:47:59.379824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:47:59.379824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:47:59.379824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:47:59.379824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:47:59.379824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:47:59.379824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:47:59.379824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:47:59.379824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:47:59.379824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:47:59.379824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:47:59.379824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:47:59.379824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:47:59.379824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:47:59.380742 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:47:59.380742 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:47:59.380742 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:47:59.380742 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:47:59.380742 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:47:59.380742 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:47:59.380742 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:47:59.380742 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:47:59.380742 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:47:59.380742 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:47:59.380742 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:47:59.380742 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:47:59.380742 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 20:47:59.380742 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:47:59.384406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 450/908] 10.222.0.48 () {56 vars in 1132 bytes} [Fri Sep 9 20:47:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:47:59.389176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819eca62-40d1-4804-b5bc-573f8bd5c15f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:59.462208 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:59.499020 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819eca62-40d1-4804-b5bc-573f8bd5c15f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:59.536487 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b0e5a5-dc76-4072-9d99-b2c602d755c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:59.537651 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b0e5a5-dc76-4072-9d99-b2c602d755c4 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:59.537987 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b0e5a5-dc76-4072-9d99-b2c602d755c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:59.538265 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b0e5a5-dc76-4072-9d99-b2c602d755c4 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:59.538947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b0e5a5-dc76-4072-9d99-b2c602d755c4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:59.539299 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b0e5a5-dc76-4072-9d99-b2c602d755c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:59.540713 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b0e5a5-dc76-4072-9d99-b2c602d755c4 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:59.545851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 459/909] 10.222.0.48 () {56 vars in 1126 bytes} [Fri Sep 9 20:47:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:59.551439 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2a041-1683-49d1-83f6-8885815bed41 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:59.657951 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2a041-1683-49d1-83f6-8885815bed41 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:59.693642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf3bbf-50d7-443b-89bf-22980f1989b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:59.694684 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf3bbf-50d7-443b-89bf-22980f1989b2 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:59.695159 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf3bbf-50d7-443b-89bf-22980f1989b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:59.695641 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf3bbf-50d7-443b-89bf-22980f1989b2 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:59.696486 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cf3bbf-50d7-443b-89bf-22980f1989b2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:59.696984 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cf3bbf-50d7-443b-89bf-22980f1989b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:59.698576 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cf3bbf-50d7-443b-89bf-22980f1989b2 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:59.699975 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:59.704913 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 451/910] 10.222.0.48 () {56 vars in 1126 bytes} [Fri Sep 9 20:47:59 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:59.715963 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b0e5a5-dc76-4072-9d99-b2c602d755c4 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:59.822091 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b0e5a5-dc76-4072-9d99-b2c602d755c4 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:47:59.851680 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:47:59.857557 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943598df-bd1e-41fd-8a95-79275390fa07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:47:59.858171 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943598df-bd1e-41fd-8a95-79275390fa07 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:47:59.858450 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943598df-bd1e-41fd-8a95-79275390fa07 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:47:59.858725 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943598df-bd1e-41fd-8a95-79275390fa07 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:47:59.859428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943598df-bd1e-41fd-8a95-79275390fa07 None admin] RBAC: Adding query filter params (interface=public, service_id=66381337f4634ba7a2e8ce603d9f7ec3, region_id=RegionOne) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:47:59.862590 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943598df-bd1e-41fd-8a95-79275390fa07 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:47:59.862590 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943598df-bd1e-41fd-8a95-79275390fa07 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:47:59.872469 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 460/911] 10.222.0.48 () {56 vars in 1251 bytes} [Fri Sep 9 20:47:59 2022] GET /identity/v3/endpoints?service_id=66381337f4634ba7a2e8ce603d9f7ec3&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:47:59.874189 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf3bbf-50d7-443b-89bf-22980f1989b2 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:47:59.984731 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf3bbf-50d7-443b-89bf-22980f1989b2 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:00.022100 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c539ab-dce7-4019-a742-382dd2dd229c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:00.022719 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c539ab-dce7-4019-a742-382dd2dd229c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:00.022997 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c539ab-dce7-4019-a742-382dd2dd229c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:00.023267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c539ab-dce7-4019-a742-382dd2dd229c None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:00.023930 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c539ab-dce7-4019-a742-382dd2dd229c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:00.025412 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c539ab-dce7-4019-a742-382dd2dd229c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:00.036385 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 452/912] 10.222.0.48 () {56 vars in 1087 bytes} [Fri Sep 9 20:47:59 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:48:00.626520 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bab74-98a1-4cc7-b3ba-4b38d89d8165 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:00.627323 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bab74-98a1-4cc7-b3ba-4b38d89d8165 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:00.627833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bab74-98a1-4cc7-b3ba-4b38d89d8165 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:00.629641 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 461/913] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:48:00.635394 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddee2cd-5646-475c-b3f0-6a6a9feab7f3 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:00.635915 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddee2cd-5646-475c-b3f0-6a6a9feab7f3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:00.636426 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddee2cd-5646-475c-b3f0-6a6a9feab7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:00.654618 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5ddee2cd-5646-475c-b3f0-6a6a9feab7f3 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:00.656172 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5ddee2cd-5646-475c-b3f0-6a6a9feab7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:00.715316 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:00.726928 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 453/914] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:48:00.864441 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd7021-0eba-433d-ab82-55275f5ba704 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:00.864976 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd7021-0eba-433d-ab82-55275f5ba704 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:00.864976 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd7021-0eba-433d-ab82-55275f5ba704 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:00.867120 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 462/915] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:48:00.872460 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af014c-94f3-4b1f-aaec-c0e61981dc92 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:00.872972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af014c-94f3-4b1f-aaec-c0e61981dc92 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:00.873452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af014c-94f3-4b1f-aaec-c0e61981dc92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:00.896777 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-74af014c-94f3-4b1f-aaec-c0e61981dc92 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:00.899477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-74af014c-94f3-4b1f-aaec-c0e61981dc92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:00.991750 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 454/916] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:48:01.020878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd7021-0eba-433d-ab82-55275f5ba704 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:01.126980 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd7021-0eba-433d-ab82-55275f5ba704 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:01.161535 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a5337-e22e-4e32-b5fa-a57bbae5b7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:01.162641 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a5337-e22e-4e32-b5fa-a57bbae5b7e9 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:01.162919 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a5337-e22e-4e32-b5fa-a57bbae5b7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:01.163188 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a5337-e22e-4e32-b5fa-a57bbae5b7e9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:01.174042 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a5337-e22e-4e32-b5fa-a57bbae5b7e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:48:01.178947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-d03a5337-e22e-4e32-b5fa-a57bbae5b7e9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 20:48:01.178947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:01.178947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:01.178947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:01.178947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:01.178947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:01.178947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:01.178947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:01.178947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:01.178947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:01.178947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:01.178947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:01.178947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:48:01.178947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:48:01.178947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:48:01.178947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:48:01.179928 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:48:01.179928 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:48:01.179928 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:48:01.179928 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:48:01.179928 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:01.179928 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:01.179928 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:01.179928 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:01.179928 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:01.179928 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:01.179928 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:01.179928 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:01.179928 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:01.179928 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:01.179928 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:01.179928 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:01.180869 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:01.180869 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:01.180869 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:01.180869 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:01.180869 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:48:01.180869 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:48:01.180869 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:48:01.180869 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:48:01.180869 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:48:01.180869 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:48:01.180869 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 20:48:01.180869 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:01.180869 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 463/917] 10.222.0.48 () {56 vars in 1120 bytes} [Fri Sep 9 20:48:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:01.188768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af014c-94f3-4b1f-aaec-c0e61981dc92 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:01.275512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a5337-e22e-4e32-b5fa-a57bbae5b7e9 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:01.316772 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af014c-94f3-4b1f-aaec-c0e61981dc92 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:01.336444 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:01.346287 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:01.359466 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109ce30-006b-4c9f-89c9-0ba7c22952f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:01.365164 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109ce30-006b-4c9f-89c9-0ba7c22952f3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:01.365164 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109ce30-006b-4c9f-89c9-0ba7c22952f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:01.365164 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109ce30-006b-4c9f-89c9-0ba7c22952f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:01.365164 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8109ce30-006b-4c9f-89c9-0ba7c22952f3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:01.365164 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8109ce30-006b-4c9f-89c9-0ba7c22952f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:01.365164 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8109ce30-006b-4c9f-89c9-0ba7c22952f3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:01.371995 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 455/918] 10.222.0.48 () {56 vars in 1115 bytes} [Fri Sep 9 20:48:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:01.373366 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40406a7d-40f2-4f71-9efa-479251f0dff5 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:01.373725 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40406a7d-40f2-4f71-9efa-479251f0dff5 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:01.374077 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40406a7d-40f2-4f71-9efa-479251f0dff5 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:01.375344 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 456/919] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:48:01.380268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40406a7d-40f2-4f71-9efa-479251f0dff5 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:01.388607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a5337-e22e-4e32-b5fa-a57bbae5b7e9 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:01.423237 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dac90e-ec39-4478-b436-50298dd13804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:01.423782 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dac90e-ec39-4478-b436-50298dd13804 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:01.424055 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dac90e-ec39-4478-b436-50298dd13804 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:01.424451 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dac90e-ec39-4478-b436-50298dd13804 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:01.425052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dac90e-ec39-4478-b436-50298dd13804 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:01.426416 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dac90e-ec39-4478-b436-50298dd13804 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:01.437573 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-62dac90e-ec39-4478-b436-50298dd13804 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 20:48:01.437573 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:01.437573 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:48:01.437573 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:48:01.437573 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:48:01.437573 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:48:01.437573 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:48:01.437573 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:48:01.437573 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 20:48:01.437573 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:01.437573 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:48:01.437573 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:01.437573 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:01.437573 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:01.437573 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:01.437573 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:01.437573 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:01.438591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:01.438591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:01.438591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:01.438591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:01.438591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:01.438591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:01.438591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:48:01.438591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:48:01.438591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:48:01.438591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:48:01.438591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:01.438591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:01.438591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:01.438591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:01.438591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:01.438591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:01.439419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:01.439419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:01.439419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:01.439419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:01.439419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:01.439419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:01.439419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:01.439419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:01.439419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:01.439419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:01.439419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:48:01.439419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:48:01.439419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 20:48:01.439419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:01.439419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 464/920] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Sep 9 20:48:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:01.440843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b33e5-280f-4970-b414-bd27f71545d1 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:01.440843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b33e5-280f-4970-b414-bd27f71545d1 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:01.440843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b33e5-280f-4970-b414-bd27f71545d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:01.459888 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-3d9b33e5-280f-4970-b414-bd27f71545d1 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:01.461105 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3d9b33e5-280f-4970-b414-bd27f71545d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:01.495017 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40406a7d-40f2-4f71-9efa-479251f0dff5 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:01.529346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addaa780-c2c8-45c2-bc2f-f462ab03caf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:01.529971 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addaa780-c2c8-45c2-bc2f-f462ab03caf5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:01.530246 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addaa780-c2c8-45c2-bc2f-f462ab03caf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:01.530701 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addaa780-c2c8-45c2-bc2f-f462ab03caf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:01.543010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 465/921] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:48:01.543010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b33e5-280f-4970-b414-bd27f71545d1 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:01.543243 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addaa780-c2c8-45c2-bc2f-f462ab03caf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:01.543243 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addaa780-c2c8-45c2-bc2f-f462ab03caf5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:01.560539 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-addaa780-c2c8-45c2-bc2f-f462ab03caf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:48:01.560539 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:01.560539 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:01.560539 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:01.560539 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:01.560539 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:01.560539 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:01.560539 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:01.560539 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:01.560539 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:01.560539 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:01.560539 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:01.560539 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:48:01.560539 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:48:01.560539 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:48:01.560539 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:48:01.561781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:01.561781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:01.561781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:01.561781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:01.561781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:01.561781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:01.561781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:01.561781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:01.561781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:01.561781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:01.561781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:01.561781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:01.561781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:01.561781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:01.561781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:01.561781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:01.561781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:48:01.562930 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:48:01.562930 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:48:01.562930 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:01.562930 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 457/922] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:48:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:01.569872 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addaa780-c2c8-45c2-bc2f-f462ab03caf5 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:01.600443 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:01.617540 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:01.660670 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b33e5-280f-4970-b414-bd27f71545d1 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:01.689331 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addaa780-c2c8-45c2-bc2f-f462ab03caf5 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:01.709825 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69e782-c03e-4aa9-bdfd-4ee02f25bcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:01.710522 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69e782-c03e-4aa9-bdfd-4ee02f25bcbd None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:01.710867 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69e782-c03e-4aa9-bdfd-4ee02f25bcbd None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:01.711201 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69e782-c03e-4aa9-bdfd-4ee02f25bcbd None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:01.712564 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc69e782-c03e-4aa9-bdfd-4ee02f25bcbd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:01.712939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc69e782-c03e-4aa9-bdfd-4ee02f25bcbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:01.714875 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc69e782-c03e-4aa9-bdfd-4ee02f25bcbd None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:01.725882 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 466/923] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Sep 9 20:48:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:01.727691 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69e782-c03e-4aa9-bdfd-4ee02f25bcbd None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:01.732917 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb1bc0-d411-471b-90c7-044aca790f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:01.732917 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb1bc0-d411-471b-90c7-044aca790f09 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:01.733289 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb1bc0-d411-471b-90c7-044aca790f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:01.733478 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb1bc0-d411-471b-90c7-044aca790f09 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:01.734133 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb1bc0-d411-471b-90c7-044aca790f09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:01.734442 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb1bc0-d411-471b-90c7-044aca790f09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:01.735865 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb1bc0-d411-471b-90c7-044aca790f09 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:01.752459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 458/924] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:48:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:48:01.754622 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb1bc0-d411-471b-90c7-044aca790f09 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:01.852115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:01.854398 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69e782-c03e-4aa9-bdfd-4ee02f25bcbd None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:01.859196 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb1bc0-d411-471b-90c7-044aca790f09 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:01.872313 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:01.895754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0fa6d1-7c1b-49d0-ae0d-1933a1734036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:01.896545 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0fa6d1-7c1b-49d0-ae0d-1933a1734036 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:01.896841 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0fa6d1-7c1b-49d0-ae0d-1933a1734036 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:01.897096 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0fa6d1-7c1b-49d0-ae0d-1933a1734036 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:01.897791 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0fa6d1-7c1b-49d0-ae0d-1933a1734036 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:01.898098 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0fa6d1-7c1b-49d0-ae0d-1933a1734036 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:01.898968 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b630a-6f5f-4abf-aada-ed4823c57a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:01.899450 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0fa6d1-7c1b-49d0-ae0d-1933a1734036 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:01.900307 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b630a-6f5f-4abf-aada-ed4823c57a62 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:01.900748 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b630a-6f5f-4abf-aada-ed4823c57a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:01.901166 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b630a-6f5f-4abf-aada-ed4823c57a62 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:01.907673 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 467/925] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Sep 9 20:48:01 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:48:01.907673 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0fa6d1-7c1b-49d0-ae0d-1933a1734036 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:01.912959 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b630a-6f5f-4abf-aada-ed4823c57a62 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:01.914420 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b630a-6f5f-4abf-aada-ed4823c57a62 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:01.926964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-3c5b630a-6f5f-4abf-aada-ed4823c57a62 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 20:48:01.926964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:01.926964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:01.926964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:01.926964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:01.926964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:01.926964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:01.926964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:01.926964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:01.926964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:01.926964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:01.926964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:01.926964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:48:01.926964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:48:01.926964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:48:01.926964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:48:01.935262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:01.935262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:01.935262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:48:01.935262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:48:01.935262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:48:01.935262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:48:01.935262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:01.935262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:01.935262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:01.935262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:01.935262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:01.935262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:01.935262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:01.935262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:01.935262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:01.935262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:01.935262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:01.935262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:01.936904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:01.936904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:01.936904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:48:01.936904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:48:01.936904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:48:01.936904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:48:01.936904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:48:01.936904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:48:01.936904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 20:48:01.936904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:01.936904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 459/926] 10.222.0.48 () {56 vars in 1117 bytes} [Fri Sep 9 20:48:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:48:01.936904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b630a-6f5f-4abf-aada-ed4823c57a62 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:02.027541 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0fa6d1-7c1b-49d0-ae0d-1933a1734036 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:02.041719 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b630a-6f5f-4abf-aada-ed4823c57a62 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:02.064043 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e184c-81e1-4000-84b4-0128b05d3d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:02.064698 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e184c-81e1-4000-84b4-0128b05d3d0f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:02.064977 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e184c-81e1-4000-84b4-0128b05d3d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:02.065248 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e184c-81e1-4000-84b4-0128b05d3d0f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:02.065893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e184c-81e1-4000-84b4-0128b05d3d0f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:02.067507 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e184c-81e1-4000-84b4-0128b05d3d0f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:02.077814 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-572e184c-81e1-4000-84b4-0128b05d3d0f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 20:48:02.077814 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:02.077814 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:48:02.077814 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:48:02.077814 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:48:02.077814 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:48:02.077814 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:48:02.077814 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:48:02.077814 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 20:48:02.077814 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:02.077814 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:48:02.077814 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:02.077814 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:02.077814 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:02.077814 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:02.077814 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:02.077814 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:02.078812 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:02.078812 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:02.078812 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:02.078812 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:02.078812 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:02.078812 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:02.078812 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:48:02.078812 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:48:02.078812 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:48:02.078812 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:48:02.078812 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:02.078812 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:02.078812 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:02.078812 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:02.078812 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:02.078812 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:02.079643 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:02.079643 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:02.079643 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:02.079643 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:02.079643 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:02.079643 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:02.079643 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:02.079643 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:02.079643 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:02.079643 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:02.079643 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:48:02.079643 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:48:02.079643 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 20:48:02.079643 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:02.080414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec3200-cd2e-4ab4-b134-9b7db65d6a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:02.080414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec3200-cd2e-4ab4-b134-9b7db65d6a48 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:02.080414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec3200-cd2e-4ab4-b134-9b7db65d6a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:02.080414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec3200-cd2e-4ab4-b134-9b7db65d6a48 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:02.080688 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec3200-cd2e-4ab4-b134-9b7db65d6a48 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:02.081315 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec3200-cd2e-4ab4-b134-9b7db65d6a48 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:02.083880 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 468/927] 10.222.0.48 () {56 vars in 1126 bytes} [Fri Sep 9 20:48:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:02.084925 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e184c-81e1-4000-84b4-0128b05d3d0f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:02.125629 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:02.168124 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 460/928] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 20:48:01 2022] POST /identity/v3/endpoints => generated 346 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:48:02.168124 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec3200-cd2e-4ab4-b134-9b7db65d6a48 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:02.189916 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e184c-81e1-4000-84b4-0128b05d3d0f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:02.192593 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:02.249566 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2299da38-1fa1-4025-b1ac-21ba92179372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:02.250250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2299da38-1fa1-4025-b1ac-21ba92179372 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:02.250818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2299da38-1fa1-4025-b1ac-21ba92179372 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:02.251097 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2299da38-1fa1-4025-b1ac-21ba92179372 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:02.252196 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2299da38-1fa1-4025-b1ac-21ba92179372 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:02.252516 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2299da38-1fa1-4025-b1ac-21ba92179372 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:02.254359 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2299da38-1fa1-4025-b1ac-21ba92179372 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:02.281087 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 469/929] 10.222.0.48 () {56 vars in 1149 bytes} [Fri Sep 9 20:48:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:02.282508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273408b-dc1c-47d5-8723-c69dc98f4c4b None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:02.282803 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273408b-dc1c-47d5-8723-c69dc98f4c4b None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:02.283077 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273408b-dc1c-47d5-8723-c69dc98f4c4b None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:02.284669 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 470/930] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:48:02.285423 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273408b-dc1c-47d5-8723-c69dc98f4c4b None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:02.314085 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec3200-cd2e-4ab4-b134-9b7db65d6a48 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:02.356455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5165a28a-ad59-41cd-9c0a-bf41e0327b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:02.357068 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165a28a-ad59-41cd-9c0a-bf41e0327b43 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:02.357344 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165a28a-ad59-41cd-9c0a-bf41e0327b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:02.357615 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165a28a-ad59-41cd-9c0a-bf41e0327b43 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:02.358270 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5165a28a-ad59-41cd-9c0a-bf41e0327b43 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:02.358575 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5165a28a-ad59-41cd-9c0a-bf41e0327b43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:02.359944 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5165a28a-ad59-41cd-9c0a-bf41e0327b43 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:02.366942 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 461/931] 10.222.0.48 () {56 vars in 1122 bytes} [Fri Sep 9 20:48:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:02.368348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acda31-303a-42ad-a2d5-3955bcf8e0c6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:02.368348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acda31-303a-42ad-a2d5-3955bcf8e0c6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:02.368529 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acda31-303a-42ad-a2d5-3955bcf8e0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:02.387397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-11acda31-303a-42ad-a2d5-3955bcf8e0c6 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:02.388723 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-11acda31-303a-42ad-a2d5-3955bcf8e0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:02.399344 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:02.412742 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273408b-dc1c-47d5-8723-c69dc98f4c4b None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:02.448970 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204dc0fc-8267-4d1e-9d43-c3c0bbebc671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:02.449490 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204dc0fc-8267-4d1e-9d43-c3c0bbebc671 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:02.449771 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204dc0fc-8267-4d1e-9d43-c3c0bbebc671 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:02.450052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204dc0fc-8267-4d1e-9d43-c3c0bbebc671 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:02.459554 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 462/932] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:48:02.460237 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11acda31-303a-42ad-a2d5-3955bcf8e0c6 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:02.465091 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204dc0fc-8267-4d1e-9d43-c3c0bbebc671 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:02.466804 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204dc0fc-8267-4d1e-9d43-c3c0bbebc671 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:02.468408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:02.488910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-204dc0fc-8267-4d1e-9d43-c3c0bbebc671 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:48:02.488910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:02.488910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:02.488910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:02.488910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:02.488910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:02.488910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:02.488910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:02.488910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:02.488910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:02.488910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:02.488910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:02.488910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:48:02.488910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:48:02.488910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:48:02.488910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:48:02.489942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:02.489942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:02.489942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:02.489942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:02.489942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:02.489942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:02.489942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:02.489942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:02.489942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:02.489942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:02.489942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:02.489942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:02.489942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:02.489942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:02.489942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:02.489942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:02.489942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:48:02.491501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:48:02.491501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:48:02.491501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:02.494025 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 471/933] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:48:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:02.499305 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204dc0fc-8267-4d1e-9d43-c3c0bbebc671 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:02.584622 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11acda31-303a-42ad-a2d5-3955bcf8e0c6 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:02.630364 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204dc0fc-8267-4d1e-9d43-c3c0bbebc671 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:02.631075 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdcb7ba-5154-46d4-a87c-d81812440899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:02.631654 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdcb7ba-5154-46d4-a87c-d81812440899 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:02.631935 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdcb7ba-5154-46d4-a87c-d81812440899 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:02.632313 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdcb7ba-5154-46d4-a87c-d81812440899 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:02.632953 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdcb7ba-5154-46d4-a87c-d81812440899 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:02.633523 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdcb7ba-5154-46d4-a87c-d81812440899 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:02.635828 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdcb7ba-5154-46d4-a87c-d81812440899 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:02.646858 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 463/934] 10.222.0.48 () {56 vars in 1122 bytes} [Fri Sep 9 20:48:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:02.649059 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdcb7ba-5154-46d4-a87c-d81812440899 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:02.677232 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed4142-95e3-4141-b7ed-a862fc04a23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:02.677880 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed4142-95e3-4141-b7ed-a862fc04a23f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:02.678161 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed4142-95e3-4141-b7ed-a862fc04a23f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:02.678435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed4142-95e3-4141-b7ed-a862fc04a23f None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:02.679355 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed4142-95e3-4141-b7ed-a862fc04a23f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:02.679660 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed4142-95e3-4141-b7ed-a862fc04a23f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:02.681059 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed4142-95e3-4141-b7ed-a862fc04a23f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:02.698583 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 472/935] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:02.703782 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed4142-95e3-4141-b7ed-a862fc04a23f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:02.715602 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:02.730295 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:02.762316 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdcb7ba-5154-46d4-a87c-d81812440899 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:02.797706 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f63056-c244-485f-b03a-7713b4bb10a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:02.798315 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f63056-c244-485f-b03a-7713b4bb10a3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:02.798597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f63056-c244-485f-b03a-7713b4bb10a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:02.798875 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f63056-c244-485f-b03a-7713b4bb10a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:02.799578 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f63056-c244-485f-b03a-7713b4bb10a3 None admin] RBAC: Adding query filter params (interface=public, service_id=855f6948d823486098361b3c3284cb7e, region_id=RegionOne) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:02.799892 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f63056-c244-485f-b03a-7713b4bb10a3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:02.801309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f63056-c244-485f-b03a-7713b4bb10a3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:02.806430 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 464/936] 10.222.0.48 () {56 vars in 1251 bytes} [Fri Sep 9 20:48:02 2022] GET /identity/v3/endpoints?service_id=855f6948d823486098361b3c3284cb7e&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:02.807020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f63056-c244-485f-b03a-7713b4bb10a3 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:02.818474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed4142-95e3-4141-b7ed-a862fc04a23f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:02.854556 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae37126e-1039-400f-aa4e-6a7ae5391c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:02.855108 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37126e-1039-400f-aa4e-6a7ae5391c9c None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:02.855383 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37126e-1039-400f-aa4e-6a7ae5391c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:02.855652 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37126e-1039-400f-aa4e-6a7ae5391c9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:02.862737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae37126e-1039-400f-aa4e-6a7ae5391c9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:02.864396 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae37126e-1039-400f-aa4e-6a7ae5391c9c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:02.877217 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-ae37126e-1039-400f-aa4e-6a7ae5391c9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:48:02.877217 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:02.877217 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:02.877217 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:02.877217 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:02.877217 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:02.877217 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:02.877217 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:02.877217 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:02.877217 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:02.877217 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:02.877217 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:02.877217 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:48:02.877217 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:48:02.877217 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:48:02.878681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:48:02.878681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:02.878681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:02.878681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:02.878681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:02.878681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:02.878681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:02.878681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:02.878681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:02.878681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:02.878681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:02.878681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:02.878681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:02.878681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:02.878681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:02.878681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:02.878681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:02.879601 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:48:02.879601 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:48:02.879601 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:48:02.879601 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:48:02.879601 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:48:02.879601 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:48:02.879601 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:48:02.879601 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:02.879601 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 473/937] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:48:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:48:02.879601 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae37126e-1039-400f-aa4e-6a7ae5391c9c None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:02.917055 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f63056-c244-485f-b03a-7713b4bb10a3 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:02.951788 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b16871-65cb-47b3-a966-525c2e61c145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:02.952451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b16871-65cb-47b3-a966-525c2e61c145 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:02.952726 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b16871-65cb-47b3-a966-525c2e61c145 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:02.952995 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b16871-65cb-47b3-a966-525c2e61c145 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:02.953575 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b16871-65cb-47b3-a966-525c2e61c145 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:02.954940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b16871-65cb-47b3-a966-525c2e61c145 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:02.956973 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:02.957157 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:02.965650 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-c4b16871-65cb-47b3-a966-525c2e61c145 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 20:48:02.965650 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:02.965650 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:48:02.965650 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:48:02.965650 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:48:02.965650 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:48:02.965650 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:48:02.965650 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:48:02.965650 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 20:48:02.965650 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:02.965650 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:48:02.965650 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:02.965650 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:02.965650 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:02.965650 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:02.965650 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:02.966644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:02.966644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:02.966644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:02.966644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:02.966644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:02.966644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:02.966644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:02.966644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:48:02.966644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:48:02.966644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:48:02.966644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:48:02.966644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:02.966644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:02.966644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:02.966644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:02.966644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:02.966644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:02.974588 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:02.974588 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:02.974588 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:02.974588 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:02.974588 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:02.974588 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:02.974588 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:02.974588 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:02.974588 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:02.974588 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:02.974588 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:48:02.974588 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:48:02.974588 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 20:48:02.974588 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:02.974588 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 465/938] 10.222.0.48 () {56 vars in 1132 bytes} [Fri Sep 9 20:48:02 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:02.975446 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b16871-65cb-47b3-a966-525c2e61c145 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:02.997933 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae37126e-1039-400f-aa4e-6a7ae5391c9c None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:03.039798 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f0e57-7a0f-476c-badc-395f66eb7cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:03.040567 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f0e57-7a0f-476c-badc-395f66eb7cee None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:03.040941 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f0e57-7a0f-476c-badc-395f66eb7cee None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:03.041312 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f0e57-7a0f-476c-badc-395f66eb7cee None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:03.042038 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f0e57-7a0f-476c-badc-395f66eb7cee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:03.043480 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f0e57-7a0f-476c-badc-395f66eb7cee None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:03.049791 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 474/939] 10.222.0.48 () {56 vars in 1087 bytes} [Fri Sep 9 20:48:02 2022] GET /identity/v3/services => generated 1497 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:48:03.050605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f0e57-7a0f-476c-badc-395f66eb7cee None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:03.098956 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b16871-65cb-47b3-a966-525c2e61c145 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:03.140641 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3028a-e3ac-4fba-8f1e-3ae9820c4096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:03.141257 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3028a-e3ac-4fba-8f1e-3ae9820c4096 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:03.141540 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3028a-e3ac-4fba-8f1e-3ae9820c4096 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:03.141806 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3028a-e3ac-4fba-8f1e-3ae9820c4096 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:03.142580 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3028a-e3ac-4fba-8f1e-3ae9820c4096 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:03.142892 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3028a-e3ac-4fba-8f1e-3ae9820c4096 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:03.144392 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3028a-e3ac-4fba-8f1e-3ae9820c4096 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:03.158583 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 466/940] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:48:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:03.164853 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3028a-e3ac-4fba-8f1e-3ae9820c4096 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:03.198125 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f0e57-7a0f-476c-badc-395f66eb7cee None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:03.234095 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:03.242308 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:03.248308 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc6993-98f4-4c9a-a7ac-2ca3eec1da1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:03.248743 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc6993-98f4-4c9a-a7ac-2ca3eec1da1b None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:03.249019 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc6993-98f4-4c9a-a7ac-2ca3eec1da1b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:03.249288 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc6993-98f4-4c9a-a7ac-2ca3eec1da1b None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:03.250096 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc6993-98f4-4c9a-a7ac-2ca3eec1da1b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:03.250228 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc6993-98f4-4c9a-a7ac-2ca3eec1da1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:03.251548 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc6993-98f4-4c9a-a7ac-2ca3eec1da1b None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:03.256887 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 475/941] 10.222.0.48 () {56 vars in 1126 bytes} [Fri Sep 9 20:48:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:03.263106 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc6993-98f4-4c9a-a7ac-2ca3eec1da1b None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:03.287194 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3028a-e3ac-4fba-8f1e-3ae9820c4096 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:03.328229 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960e57a-0ac4-4a4f-9a3e-6df153366c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:03.328907 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960e57a-0ac4-4a4f-9a3e-6df153366c38 None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:03.329184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960e57a-0ac4-4a4f-9a3e-6df153366c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:03.329455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960e57a-0ac4-4a4f-9a3e-6df153366c38 None admin] PATH_INFO: `/v3/projects/91319aa782334fb9930770029b74e84a/users/8c21ea7d3cbb494886e60a4bd4f39145/roles/63740f83debd49b6a81f640fe2a07ce2` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:03.332826 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0960e57a-0ac4-4a4f-9a3e-6df153366c38 None admin] RBAC: Authorizing `identity:create_grant(project_id=91319aa782334fb9930770029b74e84a, user_id=8c21ea7d3cbb494886e60a4bd4f39145, role_id=63740f83debd49b6a81f640fe2a07ce2)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:03.334996 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0960e57a-0ac4-4a4f-9a3e-6df153366c38 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:03.378437 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc6993-98f4-4c9a-a7ac-2ca3eec1da1b None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:03.414427 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937cc517-c193-43ae-9a5b-6916b066a0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:03.415118 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937cc517-c193-43ae-9a5b-6916b066a0de None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:03.415392 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937cc517-c193-43ae-9a5b-6916b066a0de None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:03.415660 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937cc517-c193-43ae-9a5b-6916b066a0de None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:03.416312 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937cc517-c193-43ae-9a5b-6916b066a0de None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:03.416613 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937cc517-c193-43ae-9a5b-6916b066a0de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:03.418011 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937cc517-c193-43ae-9a5b-6916b066a0de None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:03.428099 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 476/942] 10.222.0.48 () {56 vars in 1126 bytes} [Fri Sep 9 20:48:03 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:03.429372 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937cc517-c193-43ae-9a5b-6916b066a0de None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:03.438254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 467/943] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Sep 9 20:48:03 2022] PUT /identity/v3/projects/91319aa782334fb9930770029b74e84a/users/8c21ea7d3cbb494886e60a4bd4f39145/roles/63740f83debd49b6a81f640fe2a07ce2 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:48:03.502489 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:03.549643 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937cc517-c193-43ae-9a5b-6916b066a0de None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:03.588995 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a5f8e9-79de-4457-992a-83131905e7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:03.593577 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5f8e9-79de-4457-992a-83131905e7f0 None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:03.593577 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5f8e9-79de-4457-992a-83131905e7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:03.593577 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5f8e9-79de-4457-992a-83131905e7f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:03.593577 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a5f8e9-79de-4457-992a-83131905e7f0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:03.593577 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a5f8e9-79de-4457-992a-83131905e7f0 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:03.692158 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 477/944] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 20:48:03 2022] POST /identity/v3/endpoints => generated 347 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:48:04.465226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7ac83-4ae8-4cbc-860f-d2ae4fd43238 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:04.465759 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7ac83-4ae8-4cbc-860f-d2ae4fd43238 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:04.465759 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7ac83-4ae8-4cbc-860f-d2ae4fd43238 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:04.467228 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 468/945] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:48:04.472878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77693ee-a4ac-47e0-98b3-4b28f782247e None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:04.473702 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77693ee-a4ac-47e0-98b3-4b28f782247e None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:04.474200 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77693ee-a4ac-47e0-98b3-4b28f782247e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:04.495963 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-e77693ee-a4ac-47e0-98b3-4b28f782247e None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:04.497575 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e77693ee-a4ac-47e0-98b3-4b28f782247e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:04.583700 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 478/946] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:48:04.862455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7ac83-4ae8-4cbc-860f-d2ae4fd43238 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:04.965798 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7ac83-4ae8-4cbc-860f-d2ae4fd43238 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:04.989212 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:05.007383 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d51d1-2ede-4748-9645-beb44fee18c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:05.011796 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d51d1-2ede-4748-9645-beb44fee18c1 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:05.011796 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d51d1-2ede-4748-9645-beb44fee18c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:05.011796 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d51d1-2ede-4748-9645-beb44fee18c1 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:05.011796 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d51d1-2ede-4748-9645-beb44fee18c1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:05.011796 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d51d1-2ede-4748-9645-beb44fee18c1 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:05.013346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: WARNING keystone.server.flask.common [None req-be1d51d1-2ede-4748-9645-beb44fee18c1 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 09 20:48:05.050593 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.sql.core [None req-be1d51d1-2ede-4748-9645-beb44fee18c1 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 20:48:05.050593 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 09 20:48:05.050593 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [parameters: {'id': '95a46779a3554f54b53652ad35854345', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 09 20:48:05.050593 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81629) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 20:48:05.059142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-be1d51d1-2ede-4748-9645-beb44fee18c1 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 95a46779a3554f54b53652ad35854345.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 95a46779a3554f54b53652ad35854345. Sep 09 20:48:05.059142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:05.059142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 20:48:05.059142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 20:48:05.059142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 20:48:05.059142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 20:48:05.059142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 20:48:05.059142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 20:48:05.059142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 20:48:05.059142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application conn.query(q) Sep 09 20:48:05.059142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 20:48:05.059142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 20:48:05.059142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 20:48:05.067642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application result.read() Sep 09 20:48:05.067642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 20:48:05.067642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 20:48:05.067642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 20:48:05.067642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 20:48:05.067642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 20:48:05.067642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 20:48:05.067642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 20:48:05.067642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 20:48:05.067642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 20:48:05.067642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:05.067642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 20:48:05.067642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:05.067642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:05.067642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 20:48:05.067642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 20:48:05.067642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 09 20:48:05.067642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 09 20:48:05.069221 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 20:48:05.069221 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application next(self.gen) Sep 09 20:48:05.069221 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 20:48:05.069221 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application yield resource Sep 09 20:48:05.069221 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 20:48:05.069221 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application next(self.gen) Sep 09 20:48:05.069221 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 20:48:05.069221 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 20:48:05.069221 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 20:48:05.069221 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 20:48:05.069221 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 20:48:05.069221 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise self.value Sep 09 20:48:05.069221 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 20:48:05.069221 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 20:48:05.069221 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 20:48:05.069221 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application session.commit() Sep 09 20:48:05.069221 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 20:48:05.070118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 20:48:05.070118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 20:48:05.070118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 20:48:05.070118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 20:48:05.070118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self.session.flush() Sep 09 20:48:05.070118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 20:48:05.070118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 20:48:05.070118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 20:48:05.070118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 20:48:05.070118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 20:48:05.070118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application compat.raise_( Sep 09 20:48:05.070118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 20:48:05.070118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception Sep 09 20:48:05.070118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 20:48:05.070118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 20:48:05.070118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 20:48:05.070118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 20:48:05.072126 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 20:48:05.072126 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 20:48:05.072126 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 20:48:05.072126 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 20:48:05.072126 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 09 20:48:05.072126 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 09 20:48:05.072126 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 20:48:05.072126 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 20:48:05.072126 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 20:48:05.072126 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 20:48:05.072126 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 20:48:05.072126 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 20:48:05.072126 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 20:48:05.072126 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 20:48:05.072126 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 20:48:05.073979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 20:48:05.073979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 20:48:05.073979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception Sep 09 20:48:05.073979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 20:48:05.073979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 20:48:05.073979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 20:48:05.073979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 20:48:05.073979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 20:48:05.073979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 20:48:05.073979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 20:48:05.073979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application conn.query(q) Sep 09 20:48:05.073979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 20:48:05.073979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 20:48:05.073979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 20:48:05.073979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application result.read() Sep 09 20:48:05.073979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 20:48:05.073979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 20:48:05.076053 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 20:48:05.076053 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 20:48:05.076053 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 20:48:05.076053 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 20:48:05.076053 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 20:48:05.076053 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 20:48:05.076053 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 20:48:05.076053 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 09 20:48:05.076053 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [parameters: {'id': '95a46779a3554f54b53652ad35854345', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 09 20:48:05.076053 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 20:48:05.076053 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:05.076053 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:48:05.076053 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:05.076053 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:05.076053 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 09 20:48:05.077992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 09 20:48:05.077992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 20:48:05.077992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 20:48:05.077992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 09 20:48:05.077992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:05.077992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:48:05.077992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:05.077992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:05.077992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:05.077992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:05.077992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:05.077992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:05.077992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:05.077992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:05.077992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:05.077992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:05.077992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:05.080135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:05.080135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 09 20:48:05.080135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 09 20:48:05.080135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:05.080135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:05.080135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 09 20:48:05.080135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 09 20:48:05.080135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 95a46779a3554f54b53652ad35854345. Sep 09 20:48:05.080135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:05.080135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 469/947] 10.222.0.48 () {60 vars in 1140 bytes} [Fri Sep 9 20:48:04 2022] POST /identity/v3/projects => generated 262 bytes in 199 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:05.081406 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77693ee-a4ac-47e0-98b3-4b28f782247e None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:05.117617 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:05.177077 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77693ee-a4ac-47e0-98b3-4b28f782247e None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:05.212949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a42b3-5cd1-4bfe-b4da-444fd25e375f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:05.214123 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a42b3-5cd1-4bfe-b4da-444fd25e375f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:05.214607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a42b3-5cd1-4bfe-b4da-444fd25e375f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:05.215078 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a42b3-5cd1-4bfe-b4da-444fd25e375f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:05.222156 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a42b3-5cd1-4bfe-b4da-444fd25e375f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:05.223847 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a42b3-5cd1-4bfe-b4da-444fd25e375f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:05.236935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-058a42b3-5cd1-4bfe-b4da-444fd25e375f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:48:05.236935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:05.236935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:05.236935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:05.236935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:05.236935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:05.236935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:05.236935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:05.236935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:05.236935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:05.236935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:05.236935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:05.236935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:48:05.236935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:48:05.236935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:48:05.237894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:48:05.237894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:05.237894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:05.237894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:05.237894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:05.237894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:05.237894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:05.237894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:05.237894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:05.237894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:05.237894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:05.237894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:05.237894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:05.237894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:05.237894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:05.237894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:05.237894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:05.238820 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:48:05.238820 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:48:05.238820 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:48:05.238820 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:48:05.238820 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:48:05.238820 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:48:05.238820 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:48:05.238820 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:05.242082 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 479/948] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:48:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:05.250846 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d51d1-2ede-4748-9645-beb44fee18c1 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:05.359410 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d51d1-2ede-4748-9645-beb44fee18c1 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:05.360036 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edad88-b0c6-4304-b360-2efca7c0004f None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:05.360568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edad88-b0c6-4304-b360-2efca7c0004f None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:05.361065 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edad88-b0c6-4304-b360-2efca7c0004f None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:05.367052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 480/949] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:48:05.368850 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17368c3c-bda1-44a9-9729-25de3b5629f5 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:05.369351 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17368c3c-bda1-44a9-9729-25de3b5629f5 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:05.369824 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17368c3c-bda1-44a9-9729-25de3b5629f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:05.390033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-17368c3c-bda1-44a9-9729-25de3b5629f5 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:05.391553 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-17368c3c-bda1-44a9-9729-25de3b5629f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:05.396287 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c11014e-aa3c-4bfe-884b-e5509b563daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:05.396955 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11014e-aa3c-4bfe-884b-e5509b563daa None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:05.397245 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11014e-aa3c-4bfe-884b-e5509b563daa None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:05.397519 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11014e-aa3c-4bfe-884b-e5509b563daa None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:05.398186 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c11014e-aa3c-4bfe-884b-e5509b563daa None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:05.398490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c11014e-aa3c-4bfe-884b-e5509b563daa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:05.400151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c11014e-aa3c-4bfe-884b-e5509b563daa None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:05.421026 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 470/950] 10.222.0.48 () {56 vars in 1112 bytes} [Fri Sep 9 20:48:05 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:05.476221 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 481/951] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:48:05.708669 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51995efb-b026-4c79-87b4-5977552a2549 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:05.709704 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51995efb-b026-4c79-87b4-5977552a2549 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:05.709704 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51995efb-b026-4c79-87b4-5977552a2549 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:05.711295 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 471/952] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:48:05.716548 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b7bd7-9491-43dd-9046-58424a82a06f None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:05.717054 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b7bd7-9491-43dd-9046-58424a82a06f None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:05.717529 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b7bd7-9491-43dd-9046-58424a82a06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:05.736586 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-9a2b7bd7-9491-43dd-9046-58424a82a06f None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:05.738097 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-9a2b7bd7-9491-43dd-9046-58424a82a06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:05.743828 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51995efb-b026-4c79-87b4-5977552a2549 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:05.764621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:05.785242 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:05.819026 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 482/953] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:48:05.872970 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51995efb-b026-4c79-87b4-5977552a2549 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:05.912857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444451a7-d991-4758-b75f-33b5b248558b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:05.913520 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444451a7-d991-4758-b75f-33b5b248558b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:05.913799 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444451a7-d991-4758-b75f-33b5b248558b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:05.914071 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444451a7-d991-4758-b75f-33b5b248558b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:05.914677 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444451a7-d991-4758-b75f-33b5b248558b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:05.916117 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444451a7-d991-4758-b75f-33b5b248558b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:05.926409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-444451a7-d991-4758-b75f-33b5b248558b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 20:48:05.926409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:05.926409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:48:05.926409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:48:05.926409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:48:05.926409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:48:05.926409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:48:05.926409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:48:05.926409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 20:48:05.926409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:05.926409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:48:05.926409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:05.926409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:05.926409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:05.926409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:05.926409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:05.926409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:05.927626 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:05.927626 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:05.927626 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:05.927626 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:05.927626 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:05.927626 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:05.927626 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:48:05.927626 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:48:05.927626 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:48:05.927626 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:48:05.927626 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:05.927626 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:05.927626 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:05.927626 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:05.927626 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:05.927626 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:05.928566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:05.928566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:05.928566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:05.928566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:05.928566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:05.928566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:05.928566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:05.928566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:05.928566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:05.928566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:05.928566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:48:05.928566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:48:05.928566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 20:48:05.928566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:05.928566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 472/954] 10.222.0.48 () {56 vars in 1126 bytes} [Fri Sep 9 20:48:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:48:05.935767 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b7bd7-9491-43dd-9046-58424a82a06f None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:06.025388 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac51a1-8ed5-4e92-b575-2bd6862942a4 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:06.025689 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac51a1-8ed5-4e92-b575-2bd6862942a4 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:06.025938 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac51a1-8ed5-4e92-b575-2bd6862942a4 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:06.027585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 473/955] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:48:06.032474 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a95f8-e364-43b9-b561-79b41e405c75 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:06.032753 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a95f8-e364-43b9-b561-79b41e405c75 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:06.033027 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a95f8-e364-43b9-b561-79b41e405c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:06.052692 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b7bd7-9491-43dd-9046-58424a82a06f None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:06.053050 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-741a95f8-e364-43b9-b561-79b41e405c75 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:06.054742 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-741a95f8-e364-43b9-b561-79b41e405c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:06.094099 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5312470c-e1f6-4321-af42-e73922622973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:06.094710 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312470c-e1f6-4321-af42-e73922622973 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:06.094993 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312470c-e1f6-4321-af42-e73922622973 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:06.095264 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312470c-e1f6-4321-af42-e73922622973 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:06.095891 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5312470c-e1f6-4321-af42-e73922622973 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:06.096226 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5312470c-e1f6-4321-af42-e73922622973 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:06.097596 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5312470c-e1f6-4321-af42-e73922622973 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:06.104026 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 483/956] 10.222.0.48 () {56 vars in 1122 bytes} [Fri Sep 9 20:48:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:06.113445 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5312470c-e1f6-4321-af42-e73922622973 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:06.141139 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:06.142556 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 474/957] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:48:06.143318 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a95f8-e364-43b9-b561-79b41e405c75 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:06.172256 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:06.234955 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5312470c-e1f6-4321-af42-e73922622973 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:06.265221 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a95f8-e364-43b9-b561-79b41e405c75 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:06.270554 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9392d6-3ed9-4be4-ae7b-3d2a1f4c155d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:06.271272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9392d6-3ed9-4be4-ae7b-3d2a1f4c155d None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:06.271618 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9392d6-3ed9-4be4-ae7b-3d2a1f4c155d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:06.271958 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9392d6-3ed9-4be4-ae7b-3d2a1f4c155d None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:06.272682 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9392d6-3ed9-4be4-ae7b-3d2a1f4c155d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:06.273052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9392d6-3ed9-4be4-ae7b-3d2a1f4c155d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:06.274475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9392d6-3ed9-4be4-ae7b-3d2a1f4c155d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:06.279791 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 484/958] 10.222.0.48 () {56 vars in 1122 bytes} [Fri Sep 9 20:48:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:06.285750 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9392d6-3ed9-4be4-ae7b-3d2a1f4c155d None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:06.308153 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e3475-8b21-4e64-8db3-7b454e33e85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:06.308788 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e3475-8b21-4e64-8db3-7b454e33e85c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:06.309073 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e3475-8b21-4e64-8db3-7b454e33e85c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:06.309325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e3475-8b21-4e64-8db3-7b454e33e85c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:06.321092 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e3475-8b21-4e64-8db3-7b454e33e85c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:48:06.326095 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-203e3475-8b21-4e64-8db3-7b454e33e85c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 20:48:06.326095 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:06.326095 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:06.326095 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:06.326095 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:06.326095 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:06.326095 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:06.326095 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:06.326095 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:06.326095 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:06.326095 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:06.326095 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:06.326095 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:48:06.326095 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:48:06.326095 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:48:06.326095 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:48:06.327325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:48:06.327325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:48:06.327325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:48:06.327325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:48:06.327325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:06.327325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:06.327325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:06.327325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:06.327325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:06.327325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:06.327325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:06.327325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:06.327325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:06.327325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:06.327325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:06.327325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:06.333200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:06.333200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:06.333200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:06.333200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:06.333200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:48:06.333200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:48:06.333200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:48:06.333200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:48:06.333200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:48:06.333200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:48:06.333200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 20:48:06.333200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:06.333200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 475/959] 10.222.0.48 () {56 vars in 1120 bytes} [Fri Sep 9 20:48:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:06.335044 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e3475-8b21-4e64-8db3-7b454e33e85c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:06.393890 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:06.407529 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9392d6-3ed9-4be4-ae7b-3d2a1f4c155d None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:06.428708 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:06.441634 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e3475-8b21-4e64-8db3-7b454e33e85c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:06.449043 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a02e4-3c3c-4203-ae70-2914b6b17505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:06.449655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a02e4-3c3c-4203-ae70-2914b6b17505 None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:06.449929 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a02e4-3c3c-4203-ae70-2914b6b17505 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:06.450199 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a02e4-3c3c-4203-ae70-2914b6b17505 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:06.450859 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a02e4-3c3c-4203-ae70-2914b6b17505 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:06.452239 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a02e4-3c3c-4203-ae70-2914b6b17505 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:06.475790 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e7b68-6698-437f-a837-0e10eafa58d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:06.476416 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e7b68-6698-437f-a837-0e10eafa58d3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:06.476877 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e7b68-6698-437f-a837-0e10eafa58d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:06.476971 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e7b68-6698-437f-a837-0e10eafa58d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:06.477604 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e7b68-6698-437f-a837-0e10eafa58d3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:06.477903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e7b68-6698-437f-a837-0e10eafa58d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:06.479296 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e7b68-6698-437f-a837-0e10eafa58d3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:06.486982 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 476/960] 10.222.0.48 () {56 vars in 1115 bytes} [Fri Sep 9 20:48:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:06.487754 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e7b68-6698-437f-a837-0e10eafa58d3 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:06.544602 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 485/961] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 20:48:06 2022] POST /identity/v3/endpoints => generated 349 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:48:06.546370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a02e4-3c3c-4203-ae70-2914b6b17505 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:06.590849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e7b68-6698-437f-a837-0e10eafa58d3 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:06.645119 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bcf0c6-6294-4bb4-86cd-09e3c8f4cf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:06.646390 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bcf0c6-6294-4bb4-86cd-09e3c8f4cf0d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:06.646680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bcf0c6-6294-4bb4-86cd-09e3c8f4cf0d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:06.646958 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bcf0c6-6294-4bb4-86cd-09e3c8f4cf0d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:06.647848 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bcf0c6-6294-4bb4-86cd-09e3c8f4cf0d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:06.649486 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bcf0c6-6294-4bb4-86cd-09e3c8f4cf0d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:06.661714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-59bcf0c6-6294-4bb4-86cd-09e3c8f4cf0d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 20:48:06.661714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:06.661714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:48:06.661714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:48:06.661714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:48:06.661714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:48:06.661714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:48:06.661714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:48:06.661714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 20:48:06.661714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:06.661714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:48:06.661714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:06.661714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:06.661714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:06.661714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:06.661714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:06.661714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:06.663333 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:06.663333 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:06.663333 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:06.663333 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:06.663333 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:06.663333 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:06.663333 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:48:06.663333 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:48:06.663333 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:48:06.663333 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:48:06.663333 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:06.663333 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:06.663333 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:06.663333 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:06.663333 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:06.663333 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:06.664225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:06.664225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:06.664225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:06.664225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:06.664225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:06.664225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:06.664225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:06.664225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:06.664225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:06.664225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:06.664225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:48:06.664225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:48:06.664225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 20:48:06.664225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:06.665004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 477/962] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:48:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:06.670916 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bcf0c6-6294-4bb4-86cd-09e3c8f4cf0d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:06.707828 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:06.717287 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a02e4-3c3c-4203-ae70-2914b6b17505 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:06.764242 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f65aa7-5f94-4bf2-9ca2-4b41e36648bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:06.764896 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f65aa7-5f94-4bf2-9ca2-4b41e36648bb None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:06.765184 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f65aa7-5f94-4bf2-9ca2-4b41e36648bb None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:06.765459 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f65aa7-5f94-4bf2-9ca2-4b41e36648bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:06.778864 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f65aa7-5f94-4bf2-9ca2-4b41e36648bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:06.780353 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f65aa7-5f94-4bf2-9ca2-4b41e36648bb None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:06.783212 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:06.799614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-03f65aa7-5f94-4bf2-9ca2-4b41e36648bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:48:06.799614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:06.799614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:06.799614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:06.799614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:06.799614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:06.799614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:06.799614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:06.799614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:06.799614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:06.799614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:06.799614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:06.799614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:48:06.799614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:48:06.799614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:48:06.799614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:48:06.800626 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:06.800626 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:06.800626 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:06.800626 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:06.800626 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:06.800626 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:06.800626 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:06.800626 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:06.800626 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:06.800626 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:06.800626 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:06.800626 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:06.800626 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:06.800626 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:06.800626 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:06.800626 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:06.800626 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:48:06.801541 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:48:06.801541 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:48:06.801541 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:06.801541 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 486/963] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:48:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:06.806554 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f65aa7-5f94-4bf2-9ca2-4b41e36648bb None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:06.811783 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bcf0c6-6294-4bb4-86cd-09e3c8f4cf0d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:06.860378 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fa604-68b6-4c9e-bedd-e453570a9fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:06.861399 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fa604-68b6-4c9e-bedd-e453570a9fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:06.861888 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fa604-68b6-4c9e-bedd-e453570a9fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:06.862388 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fa604-68b6-4c9e-bedd-e453570a9fc0 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:06.863242 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4fa604-68b6-4c9e-bedd-e453570a9fc0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:06.864336 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4fa604-68b6-4c9e-bedd-e453570a9fc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:06.866952 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4fa604-68b6-4c9e-bedd-e453570a9fc0 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:06.874670 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 478/964] 10.222.0.48 () {56 vars in 1112 bytes} [Fri Sep 9 20:48:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:06.880113 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fa604-68b6-4c9e-bedd-e453570a9fc0 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:06.936812 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f65aa7-5f94-4bf2-9ca2-4b41e36648bb None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:06.968306 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:06.996170 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3923b-6e12-4bfe-b28f-74536b26a0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:06.996985 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3923b-6e12-4bfe-b28f-74536b26a0f5 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:06.997657 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3923b-6e12-4bfe-b28f-74536b26a0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:06.997936 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3923b-6e12-4bfe-b28f-74536b26a0f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:06.998888 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a3923b-6e12-4bfe-b28f-74536b26a0f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:06.999195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a3923b-6e12-4bfe-b28f-74536b26a0f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:07.001123 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a3923b-6e12-4bfe-b28f-74536b26a0f5 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:07.007990 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fa604-68b6-4c9e-bedd-e453570a9fc0 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:07.020227 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 487/965] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:48:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:48:07.025700 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3923b-6e12-4bfe-b28f-74536b26a0f5 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:07.049890 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5dab09-c891-4b31-8121-2528090abb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:07.050901 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dab09-c891-4b31-8121-2528090abb2d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:07.051380 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dab09-c891-4b31-8121-2528090abb2d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:07.051834 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dab09-c891-4b31-8121-2528090abb2d None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:07.052719 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5dab09-c891-4b31-8121-2528090abb2d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:07.053209 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5dab09-c891-4b31-8121-2528090abb2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:07.054889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5dab09-c891-4b31-8121-2528090abb2d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:07.060461 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 479/966] 10.222.0.48 () {56 vars in 1112 bytes} [Fri Sep 9 20:48:06 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:48:07.103617 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:07.150374 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3923b-6e12-4bfe-b28f-74536b26a0f5 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:07.189356 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1eb7a-3413-4306-af90-a4f0964c20e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:07.190263 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1eb7a-3413-4306-af90-a4f0964c20e3 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:07.190717 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1eb7a-3413-4306-af90-a4f0964c20e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:07.191150 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1eb7a-3413-4306-af90-a4f0964c20e3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:07.199640 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c1eb7a-3413-4306-af90-a4f0964c20e3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:07.201371 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c1eb7a-3413-4306-af90-a4f0964c20e3 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:07.214851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-e6c1eb7a-3413-4306-af90-a4f0964c20e3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 20:48:07.214851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:07.214851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:07.214851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:07.214851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:07.214851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:07.214851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:07.214851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:07.214851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:07.214851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:07.214851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:07.214851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:07.214851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:48:07.214851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:48:07.214851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:48:07.214851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:48:07.216056 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:07.216056 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:07.216056 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:48:07.216056 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:48:07.216056 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:48:07.216056 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:48:07.216056 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:07.216056 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:07.216056 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:07.216056 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:07.216056 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:07.216056 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:07.216056 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:07.216056 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:07.216056 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:07.216056 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:07.216056 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:07.216056 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:07.217235 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:07.217235 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:07.217235 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:48:07.217235 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:48:07.217235 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:48:07.217235 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:48:07.217235 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:48:07.217235 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:48:07.217235 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 20:48:07.217235 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:07.219772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 488/967] 10.222.0.48 () {56 vars in 1117 bytes} [Fri Sep 9 20:48:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:48:07.226073 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5dab09-c891-4b31-8121-2528090abb2d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:07.333429 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5dab09-c891-4b31-8121-2528090abb2d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:07.370532 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de86cbc1-7900-49a3-90d4-2248073f8670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:07.371413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de86cbc1-7900-49a3-90d4-2248073f8670 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:07.371921 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de86cbc1-7900-49a3-90d4-2248073f8670 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:07.372364 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de86cbc1-7900-49a3-90d4-2248073f8670 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:07.373234 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de86cbc1-7900-49a3-90d4-2248073f8670 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:07.373687 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de86cbc1-7900-49a3-90d4-2248073f8670 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:07.375266 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de86cbc1-7900-49a3-90d4-2248073f8670 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:07.384305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:07.394258 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 480/968] 10.222.0.48 () {56 vars in 1149 bytes} [Fri Sep 9 20:48:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:07.400489 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1eb7a-3413-4306-af90-a4f0964c20e3 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:07.501594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1eb7a-3413-4306-af90-a4f0964c20e3 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:07.523485 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:07.542997 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d29ab7-6f42-4c86-a386-0cd84edec810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:07.543612 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d29ab7-6f42-4c86-a386-0cd84edec810 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:07.543886 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d29ab7-6f42-4c86-a386-0cd84edec810 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:07.544213 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d29ab7-6f42-4c86-a386-0cd84edec810 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:07.620339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d29ab7-6f42-4c86-a386-0cd84edec810 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:07.621829 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d29ab7-6f42-4c86-a386-0cd84edec810 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:07.711560 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8901b-d2d1-4dc6-ac4c-771fc95f3a47 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:07.711907 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8901b-d2d1-4dc6-ac4c-771fc95f3a47 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:07.712138 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8901b-d2d1-4dc6-ac4c-771fc95f3a47 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:07.713618 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 481/969] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:48:07.718509 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751393b3-a8da-4bc4-baff-6b80930d0e23 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:07.718798 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751393b3-a8da-4bc4-baff-6b80930d0e23 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:07.719078 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751393b3-a8da-4bc4-baff-6b80930d0e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:07.722373 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 489/970] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:48:07 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:48:07.728831 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d29ab7-6f42-4c86-a386-0cd84edec810 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:07.743214 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-751393b3-a8da-4bc4-baff-6b80930d0e23 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:07.744651 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-751393b3-a8da-4bc4-baff-6b80930d0e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:07.748678 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:07.832110 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 482/971] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:48:07.846261 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d29ab7-6f42-4c86-a386-0cd84edec810 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:07.882620 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e783d-ce96-4188-be41-9629a1ea98f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:07.883176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e783d-ce96-4188-be41-9629a1ea98f0 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:07.883456 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e783d-ce96-4188-be41-9629a1ea98f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:07.883729 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e783d-ce96-4188-be41-9629a1ea98f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:07.894352 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860e783d-ce96-4188-be41-9629a1ea98f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:07.895869 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860e783d-ce96-4188-be41-9629a1ea98f0 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:07.912585 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-860e783d-ce96-4188-be41-9629a1ea98f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:48:07.912585 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:07.912585 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:07.912585 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:07.912585 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:07.912585 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:07.912585 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:07.912585 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:07.912585 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:07.912585 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:07.912585 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:07.912585 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:07.912585 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:48:07.912585 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:48:07.912585 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:48:07.912585 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:48:07.913591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:07.913591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:07.913591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:07.913591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:07.913591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:07.913591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:07.913591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:07.913591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:07.913591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:07.913591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:07.913591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:07.913591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:07.913591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:07.913591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:07.913591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:07.913591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:07.913591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:48:07.914506 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:48:07.914506 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:48:07.914506 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:07.914506 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 490/972] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:48:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:07.919679 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751393b3-a8da-4bc4-baff-6b80930d0e23 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:08.041390 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751393b3-a8da-4bc4-baff-6b80930d0e23 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:08.070999 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:08.081262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89008fcb-88c1-4c9a-b7a3-d2bf3ff4fa06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:08.081995 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89008fcb-88c1-4c9a-b7a3-d2bf3ff4fa06 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:08.082283 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89008fcb-88c1-4c9a-b7a3-d2bf3ff4fa06 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:08.082560 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89008fcb-88c1-4c9a-b7a3-d2bf3ff4fa06 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:08.083219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89008fcb-88c1-4c9a-b7a3-d2bf3ff4fa06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:08.083527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89008fcb-88c1-4c9a-b7a3-d2bf3ff4fa06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:08.085487 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89008fcb-88c1-4c9a-b7a3-d2bf3ff4fa06 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:08.098084 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 483/973] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:48:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:08.105105 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e783d-ce96-4188-be41-9629a1ea98f0 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:08.133888 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89008fcb-88c1-4c9a-b7a3-d2bf3ff4fa06 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:08.177785 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:08.215076 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e783d-ce96-4188-be41-9629a1ea98f0 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:08.241475 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89008fcb-88c1-4c9a-b7a3-d2bf3ff4fa06 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:08.251273 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769e4fb3-7080-44c9-8404-c53447e0814f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:08.251985 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769e4fb3-7080-44c9-8404-c53447e0814f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:08.252360 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769e4fb3-7080-44c9-8404-c53447e0814f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:08.252701 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769e4fb3-7080-44c9-8404-c53447e0814f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:08.262001 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769e4fb3-7080-44c9-8404-c53447e0814f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:08.263445 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769e4fb3-7080-44c9-8404-c53447e0814f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:08.277866 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-769e4fb3-7080-44c9-8404-c53447e0814f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:48:08.277866 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:08.277866 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:08.277866 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:08.277866 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:08.277866 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:08.277866 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:08.277866 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:08.277866 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:08.277866 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:08.277866 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:08.277866 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:08.277866 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:48:08.277866 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:48:08.277866 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:48:08.278777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:48:08.278777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:08.278777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:08.278777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:08.278777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:08.278777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:08.278777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:08.278777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:08.278777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:08.278777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:08.278777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:08.278777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:08.278777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:08.278777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:08.278777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:08.278777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:08.278777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:08.279690 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:48:08.279690 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:48:08.279690 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:48:08.279690 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:48:08.279690 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:48:08.279690 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:48:08.279690 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:48:08.279690 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:08.283920 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 491/974] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:48:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:08.284871 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769e4fb3-7080-44c9-8404-c53447e0814f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:08.288648 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b46ead-cc54-4a36-a641-4bd9a7f1ba60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:08.289263 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b46ead-cc54-4a36-a641-4bd9a7f1ba60 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:08.289541 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b46ead-cc54-4a36-a641-4bd9a7f1ba60 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:08.289816 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b46ead-cc54-4a36-a641-4bd9a7f1ba60 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:08.290508 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b46ead-cc54-4a36-a641-4bd9a7f1ba60 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:08.291955 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b46ead-cc54-4a36-a641-4bd9a7f1ba60 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:08.341781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.sql.core [None req-d4b46ead-cc54-4a36-a641-4bd9a7f1ba60 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 20:48:08.341781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 09 20:48:08.341781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [parameters: {'user_id': 'f8f736c043eb448a9a537a6b78332499', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 09 20:48:08.341781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81629) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 20:48:08.350289 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-d4b46ead-cc54-4a36-a641-4bd9a7f1ba60 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 09 20:48:08.350289 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:08.350289 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 20:48:08.350289 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 20:48:08.350289 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 20:48:08.350289 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 20:48:08.350289 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 20:48:08.350289 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 20:48:08.350289 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 20:48:08.350289 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application conn.query(q) Sep 09 20:48:08.350289 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 20:48:08.350289 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 20:48:08.350289 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 20:48:08.350289 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application result.read() Sep 09 20:48:08.350289 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 20:48:08.353121 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 20:48:08.353121 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 20:48:08.353121 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 20:48:08.353121 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 20:48:08.353121 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 20:48:08.353121 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 20:48:08.353121 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 20:48:08.353121 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 20:48:08.353121 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:08.353121 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 20:48:08.353121 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:08.353121 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:08.353121 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 20:48:08.353121 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 20:48:08.353121 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 09 20:48:08.353121 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 09 20:48:08.353121 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 20:48:08.353121 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application next(self.gen) Sep 09 20:48:08.355422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 20:48:08.355422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application yield resource Sep 09 20:48:08.355422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 20:48:08.355422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application next(self.gen) Sep 09 20:48:08.355422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 20:48:08.355422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 20:48:08.355422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 20:48:08.355422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 20:48:08.355422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 20:48:08.355422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise self.value Sep 09 20:48:08.355422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 20:48:08.355422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 20:48:08.355422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 20:48:08.355422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application session.commit() Sep 09 20:48:08.355422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 20:48:08.355422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 20:48:08.355422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 20:48:08.357640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 20:48:08.357640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 20:48:08.357640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self.session.flush() Sep 09 20:48:08.357640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 20:48:08.357640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 20:48:08.357640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 20:48:08.357640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 20:48:08.357640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 20:48:08.357640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application compat.raise_( Sep 09 20:48:08.357640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 20:48:08.357640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception Sep 09 20:48:08.357640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 20:48:08.357640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 20:48:08.357640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 20:48:08.357640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 20:48:08.357640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 20:48:08.357640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 20:48:08.364641 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 20:48:08.364641 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 20:48:08.364641 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 09 20:48:08.364641 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 09 20:48:08.364641 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 20:48:08.364641 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 20:48:08.364641 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 20:48:08.364641 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 20:48:08.364641 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 20:48:08.364641 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 20:48:08.364641 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 20:48:08.364641 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 20:48:08.364641 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 20:48:08.364641 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 20:48:08.364641 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 20:48:08.364641 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception Sep 09 20:48:08.365584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 20:48:08.365584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 20:48:08.365584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 20:48:08.365584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 20:48:08.365584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 20:48:08.365584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 20:48:08.365584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 20:48:08.365584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application conn.query(q) Sep 09 20:48:08.365584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 20:48:08.365584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 20:48:08.365584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 20:48:08.365584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application result.read() Sep 09 20:48:08.365584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 20:48:08.365584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 20:48:08.365584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 20:48:08.365584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 20:48:08.366504 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 20:48:08.366504 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 20:48:08.366504 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 20:48:08.366504 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 20:48:08.366504 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 20:48:08.366504 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 09 20:48:08.366504 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [parameters: {'user_id': 'f8f736c043eb448a9a537a6b78332499', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 09 20:48:08.366504 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 20:48:08.366504 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:08.366504 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:48:08.366504 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:08.366504 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:08.366504 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:08.366504 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:08.366504 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:08.367353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:08.367353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:08.367353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:08.367353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:08.367353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:08.367353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:08.367353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:08.367353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 09 20:48:08.367353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 09 20:48:08.367353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:08.367353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:08.367353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:48:08.367353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:48:08.367353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:48:08.367353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:48:08.367353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 09 20:48:08.367353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 09 20:48:08.369291 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 09 20:48:08.369291 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 09 20:48:08.369291 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 20:48:08.369291 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 20:48:08.369291 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 09 20:48:08.369291 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:08.369291 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 484/975] 10.222.0.48 () {60 vars in 1131 bytes} [Fri Sep 9 20:48:08 2022] POST /identity/v3/users => generated 160 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:48:08.369291 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b46ead-cc54-4a36-a641-4bd9a7f1ba60 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:08.393310 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769e4fb3-7080-44c9-8404-c53447e0814f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:08.422191 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:08.430205 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:08.431689 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6dbb4-d4ed-411e-a9db-e9e1f4b870d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:08.432415 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6dbb4-d4ed-411e-a9db-e9e1f4b870d0 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:08.432768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6dbb4-d4ed-411e-a9db-e9e1f4b870d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:08.433118 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6dbb4-d4ed-411e-a9db-e9e1f4b870d0 None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:08.433897 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af6dbb4-d4ed-411e-a9db-e9e1f4b870d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:08.434279 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af6dbb4-d4ed-411e-a9db-e9e1f4b870d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:08.435794 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af6dbb4-d4ed-411e-a9db-e9e1f4b870d0 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:08.456462 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 492/976] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:48:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:08.459514 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6dbb4-d4ed-411e-a9db-e9e1f4b870d0 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:08.503853 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b46ead-cc54-4a36-a641-4bd9a7f1ba60 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:08.542886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231b072-6135-4ee9-9c95-5dfd2e931f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:08.543624 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231b072-6135-4ee9-9c95-5dfd2e931f4e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:08.544374 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231b072-6135-4ee9-9c95-5dfd2e931f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:08.544374 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231b072-6135-4ee9-9c95-5dfd2e931f4e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:08.554196 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231b072-6135-4ee9-9c95-5dfd2e931f4e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:08.555922 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231b072-6135-4ee9-9c95-5dfd2e931f4e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:08.567867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-9231b072-6135-4ee9-9c95-5dfd2e931f4e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 20:48:08.567867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:08.567867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:08.567867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:08.567867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:08.567867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:08.567867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:08.567867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:08.567867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:08.567867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:08.567867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:08.567867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:08.567867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:48:08.567867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:48:08.567867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:48:08.567867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:48:08.569406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:08.569406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:08.569406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:48:08.569406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:48:08.569406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:48:08.569406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:48:08.569406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:08.569406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:08.569406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:08.569406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:08.569406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:08.569406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:08.569406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:08.569406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:08.569406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:08.569406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:08.569406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:08.569406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:08.570383 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:08.570383 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:08.570383 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:48:08.570383 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:48:08.570383 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:48:08.570383 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:48:08.570383 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:48:08.570383 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:48:08.570383 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 20:48:08.570383 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:08.574790 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 485/977] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:48:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:48:08.576086 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231b072-6135-4ee9-9c95-5dfd2e931f4e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:08.595402 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6dbb4-d4ed-411e-a9db-e9e1f4b870d0 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:08.641951 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ffc525-6ca4-448f-be99-ed8c7156a56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:08.642656 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ffc525-6ca4-448f-be99-ed8c7156a56e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:08.643032 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ffc525-6ca4-448f-be99-ed8c7156a56e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:08.643442 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ffc525-6ca4-448f-be99-ed8c7156a56e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:08.644718 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ffc525-6ca4-448f-be99-ed8c7156a56e None admin] RBAC: Adding query filter params (role.id=63740f83debd49b6a81f640fe2a07ce2, scope.project.id=91319aa782334fb9930770029b74e84a, user.id=8c21ea7d3cbb494886e60a4bd4f39145) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:08.645032 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ffc525-6ca4-448f-be99-ed8c7156a56e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:08.646543 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ffc525-6ca4-448f-be99-ed8c7156a56e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:08.655347 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 493/978] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:48:08 2022] GET /identity/v3/role_assignments?user.id=8c21ea7d3cbb494886e60a4bd4f39145&scope.project.id=91319aa782334fb9930770029b74e84a&role.id=63740f83debd49b6a81f640fe2a07ce2 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:08.692570 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231b072-6135-4ee9-9c95-5dfd2e931f4e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:08.706322 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:08.731623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819350b8-067c-4955-8047-543d03c6a030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:08.732273 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819350b8-067c-4955-8047-543d03c6a030 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:08.732554 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819350b8-067c-4955-8047-543d03c6a030 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:08.732830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819350b8-067c-4955-8047-543d03c6a030 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:08.733539 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819350b8-067c-4955-8047-543d03c6a030 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:08.733840 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819350b8-067c-4955-8047-543d03c6a030 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:08.735312 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819350b8-067c-4955-8047-543d03c6a030 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:08.779097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 486/979] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Sep 9 20:48:08 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:08.920766 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edaf275-b5f3-429b-8403-0a0ca1556071 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:08.921146 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edaf275-b5f3-429b-8403-0a0ca1556071 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:08.921353 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edaf275-b5f3-429b-8403-0a0ca1556071 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:08.922968 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 494/980] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:48:08.929401 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda13ca5-c937-494c-8bc9-62a080e0b67c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:08.929915 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda13ca5-c937-494c-8bc9-62a080e0b67c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:08.929992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda13ca5-c937-494c-8bc9-62a080e0b67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:08.949941 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-cda13ca5-c937-494c-8bc9-62a080e0b67c None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:08.951663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cda13ca5-c937-494c-8bc9-62a080e0b67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:09.033037 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 487/981] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:48:09.310255 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edaf275-b5f3-429b-8403-0a0ca1556071 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:09.366780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:09.384816 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd73039-f760-4adf-82f7-1db81698bb88 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:09.385089 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd73039-f760-4adf-82f7-1db81698bb88 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:09.385359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd73039-f760-4adf-82f7-1db81698bb88 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:09.386764 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 488/982] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:48:09.391540 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312a66c-1181-493d-8de9-d43019b755d7 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:09.391817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312a66c-1181-493d-8de9-d43019b755d7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:09.392092 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312a66c-1181-493d-8de9-d43019b755d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:09.411413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1312a66c-1181-493d-8de9-d43019b755d7 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:09.413220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1312a66c-1181-493d-8de9-d43019b755d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:09.416894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edaf275-b5f3-429b-8403-0a0ca1556071 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:09.454256 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a13b9-a122-443b-a850-f04477c06809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:09.454893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a13b9-a122-443b-a850-f04477c06809 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:09.455172 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a13b9-a122-443b-a850-f04477c06809 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:09.455444 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a13b9-a122-443b-a850-f04477c06809 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:09.456031 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a13b9-a122-443b-a850-f04477c06809 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:09.457421 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a13b9-a122-443b-a850-f04477c06809 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:09.466875 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-496a13b9-a122-443b-a850-f04477c06809 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 20:48:09.466875 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:09.466875 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:48:09.466875 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:48:09.466875 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:48:09.466875 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:48:09.466875 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:48:09.466875 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:48:09.466875 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 20:48:09.466875 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:09.466875 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:48:09.466875 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:09.466875 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:09.466875 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:09.466875 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:09.466875 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:09.466875 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:09.467896 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:09.467896 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:09.467896 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:09.467896 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:09.467896 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:09.467896 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:09.467896 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:48:09.467896 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:48:09.467896 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:48:09.467896 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:48:09.467896 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:09.467896 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:09.467896 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:09.467896 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:09.467896 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:09.467896 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:09.468779 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:09.468779 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:09.468779 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:09.468779 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:09.468779 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:09.468779 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:09.468779 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:09.468779 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:09.468779 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:09.468779 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:09.468779 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:48:09.468779 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:48:09.468779 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 20:48:09.468779 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:09.468779 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 495/983] 10.222.0.48 () {56 vars in 1126 bytes} [Fri Sep 9 20:48:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:09.473786 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a13b9-a122-443b-a850-f04477c06809 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:09.489046 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 489/984] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:48:09.581894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a13b9-a122-443b-a850-f04477c06809 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:09.598171 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:09.618403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a933ab9-ab44-4c22-88b0-5b0a70bdd207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:09.619208 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a933ab9-ab44-4c22-88b0-5b0a70bdd207 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:09.619505 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a933ab9-ab44-4c22-88b0-5b0a70bdd207 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:09.619782 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a933ab9-ab44-4c22-88b0-5b0a70bdd207 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:09.620462 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a933ab9-ab44-4c22-88b0-5b0a70bdd207 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:09.620815 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a933ab9-ab44-4c22-88b0-5b0a70bdd207 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:09.622147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a933ab9-ab44-4c22-88b0-5b0a70bdd207 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:09.631192 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 496/985] 10.222.0.48 () {56 vars in 1122 bytes} [Fri Sep 9 20:48:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:48:09.633999 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1312a66c-1181-493d-8de9-d43019b755d7 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:09.662404 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:09.735536 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1312a66c-1181-493d-8de9-d43019b755d7 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:09.754660 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a933ab9-ab44-4c22-88b0-5b0a70bdd207 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:09.774167 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe2ffb-8100-4610-a052-9451efd91c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:09.775060 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe2ffb-8100-4610-a052-9451efd91c23 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:09.775348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe2ffb-8100-4610-a052-9451efd91c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:09.775622 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe2ffb-8100-4610-a052-9451efd91c23 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:09.776290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fe2ffb-8100-4610-a052-9451efd91c23 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:09.776600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fe2ffb-8100-4610-a052-9451efd91c23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:09.778391 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fe2ffb-8100-4610-a052-9451efd91c23 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:09.788231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 490/986] 10.222.0.48 () {56 vars in 1122 bytes} [Fri Sep 9 20:48:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:09.791368 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe2ffb-8100-4610-a052-9451efd91c23 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:09.866421 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a933ab9-ab44-4c22-88b0-5b0a70bdd207 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:09.886544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:09.896823 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe2ffb-8100-4610-a052-9451efd91c23 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:09.902707 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de9d7ca-0f49-4776-a3db-445097afe5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:09.903496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9d7ca-0f49-4776-a3db-445097afe5d9 None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:09.903948 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9d7ca-0f49-4776-a3db-445097afe5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:09.904404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9d7ca-0f49-4776-a3db-445097afe5d9 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:09.905228 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de9d7ca-0f49-4776-a3db-445097afe5d9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:09.906642 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de9d7ca-0f49-4776-a3db-445097afe5d9 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:09.931560 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c80d0-ad0a-48c7-83e0-f717d113e029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:09.932246 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c80d0-ad0a-48c7-83e0-f717d113e029 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:09.932527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c80d0-ad0a-48c7-83e0-f717d113e029 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:09.932794 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c80d0-ad0a-48c7-83e0-f717d113e029 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:09.933494 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c80d0-ad0a-48c7-83e0-f717d113e029 None admin] RBAC: Adding query filter params (interface=admin, service_id=855f6948d823486098361b3c3284cb7e, region_id=RegionOne) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:09.933803 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c80d0-ad0a-48c7-83e0-f717d113e029 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:09.935154 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c80d0-ad0a-48c7-83e0-f717d113e029 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:09.940314 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 491/987] 10.222.0.48 () {56 vars in 1249 bytes} [Fri Sep 9 20:48:09 2022] GET /identity/v3/endpoints?service_id=855f6948d823486098361b3c3284cb7e&interface=admin®ion_id=RegionOne => generated 193 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:09.946357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c80d0-ad0a-48c7-83e0-f717d113e029 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:09.988431 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 497/988] 10.222.0.48 () {60 vars in 1141 bytes} [Fri Sep 9 20:48:09 2022] POST /identity/v3/services => generated 242 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:48:10.073705 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c80d0-ad0a-48c7-83e0-f717d113e029 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:10.117776 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4ed43-60fb-411c-ad4e-8943341a12c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:10.118313 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4ed43-60fb-411c-ad4e-8943341a12c9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:10.118592 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4ed43-60fb-411c-ad4e-8943341a12c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:10.118863 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4ed43-60fb-411c-ad4e-8943341a12c9 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:10.119473 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f4ed43-60fb-411c-ad4e-8943341a12c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:10.122634 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f4ed43-60fb-411c-ad4e-8943341a12c9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:10.136348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 492/989] 10.222.0.48 () {56 vars in 1087 bytes} [Fri Sep 9 20:48:09 2022] GET /identity/v3/services => generated 1727 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:48:10.902671 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c657a-85b9-4c82-adb4-89b4fedc53a0 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:10.903201 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c657a-85b9-4c82-adb4-89b4fedc53a0 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:10.903201 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c657a-85b9-4c82-adb4-89b4fedc53a0 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:10.904605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 498/990] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:48:10.909806 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9846b07-a60b-4aa5-a399-7ca122f587fc None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:10.910094 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9846b07-a60b-4aa5-a399-7ca122f587fc None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:10.910367 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9846b07-a60b-4aa5-a399-7ca122f587fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:10.928942 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f9846b07-a60b-4aa5-a399-7ca122f587fc None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:10.930120 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f9846b07-a60b-4aa5-a399-7ca122f587fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:10.939557 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:11.002999 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 493/991] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:48:11.270270 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c657a-85b9-4c82-adb4-89b4fedc53a0 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:11.313499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:11.372461 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c657a-85b9-4c82-adb4-89b4fedc53a0 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:11.407279 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba1730-e8b0-402e-a712-f9646d649d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:11.407914 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba1730-e8b0-402e-a712-f9646d649d70 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:11.408222 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba1730-e8b0-402e-a712-f9646d649d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:11.408498 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba1730-e8b0-402e-a712-f9646d649d70 None admin] PATH_INFO: `/v3/services/image` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:11.409083 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba1730-e8b0-402e-a712-f9646d649d70 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:11.410437 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba1730-e8b0-402e-a712-f9646d649d70 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:11.419122 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-dcba1730-e8b0-402e-a712-f9646d649d70 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 20:48:11.419122 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:11.419122 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:48:11.419122 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:48:11.419122 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:48:11.419122 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:48:11.419122 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:48:11.419122 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:48:11.419122 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 20:48:11.419122 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:11.419122 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:48:11.419122 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:11.419122 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:11.419122 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:11.419122 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:11.419122 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:11.419122 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:11.420278 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:11.420278 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:11.420278 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:11.420278 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:11.420278 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:11.420278 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:11.420278 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:48:11.420278 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:48:11.420278 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:48:11.420278 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:48:11.420278 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:11.420278 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:11.420278 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:11.420278 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:11.420278 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:11.420278 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:11.421140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:11.421140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:11.421140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:11.421140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:11.421140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:11.421140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:11.421140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:11.421140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:11.421140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:11.421140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:11.421140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:48:11.421140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:48:11.421140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 20:48:11.421140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:11.421140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 499/992] 10.222.0.48 () {56 vars in 1105 bytes} [Fri Sep 9 20:48:11 2022] GET /identity/v3/services/image => generated 86 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:11.429144 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9846b07-a60b-4aa5-a399-7ca122f587fc None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:11.528252 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9846b07-a60b-4aa5-a399-7ca122f587fc None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:11.561852 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477af8f-b2eb-4f92-bd88-ed14bade0dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:11.562453 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477af8f-b2eb-4f92-bd88-ed14bade0dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:11.562727 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477af8f-b2eb-4f92-bd88-ed14bade0dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:11.562994 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477af8f-b2eb-4f92-bd88-ed14bade0dc9 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:11.563611 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e477af8f-b2eb-4f92-bd88-ed14bade0dc9 None admin] RBAC: Adding query filter params (name=image) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:11.563909 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e477af8f-b2eb-4f92-bd88-ed14bade0dc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:11.565333 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e477af8f-b2eb-4f92-bd88-ed14bade0dc9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:11.570796 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 494/993] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:48:11 2022] GET /identity/v3/services?name=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:11.576244 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba1730-e8b0-402e-a712-f9646d649d70 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:11.672702 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba1730-e8b0-402e-a712-f9646d649d70 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:11.675006 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:11.707682 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b61ca6d-e68a-4529-8cfd-173e5d5dadeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:11.708607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61ca6d-e68a-4529-8cfd-173e5d5dadeb None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:11.708906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61ca6d-e68a-4529-8cfd-173e5d5dadeb None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:11.709173 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61ca6d-e68a-4529-8cfd-173e5d5dadeb None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:11.709800 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b61ca6d-e68a-4529-8cfd-173e5d5dadeb None admin] RBAC: Adding query filter params (type=image) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:11.710100 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b61ca6d-e68a-4529-8cfd-173e5d5dadeb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:11.711452 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b61ca6d-e68a-4529-8cfd-173e5d5dadeb None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:11.724117 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 500/994] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:48:11 2022] GET /identity/v3/services?type=image => generated 122 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:12.162358 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c3598-b076-41e7-b316-380549c95c13 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:12.162899 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c3598-b076-41e7-b316-380549c95c13 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:12.162899 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c3598-b076-41e7-b316-380549c95c13 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:12.164278 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 495/995] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:48:12.169289 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68545d7d-1bad-49b1-8077-d802d5d14388 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:12.169676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68545d7d-1bad-49b1-8077-d802d5d14388 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:12.169833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68545d7d-1bad-49b1-8077-d802d5d14388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:12.189594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-68545d7d-1bad-49b1-8077-d802d5d14388 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:12.190818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-68545d7d-1bad-49b1-8077-d802d5d14388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:12.261432 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 501/996] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:12 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:48:12.314568 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2935393-3a5e-4114-b287-4cb5b57249dd None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:12.314748 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2935393-3a5e-4114-b287-4cb5b57249dd None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:12.315022 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2935393-3a5e-4114-b287-4cb5b57249dd None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:12.316514 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 496/997] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:48:12.321685 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04caab31-b1da-4d8b-838c-bab1a23945af None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:12.321974 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04caab31-b1da-4d8b-838c-bab1a23945af None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:12.322244 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04caab31-b1da-4d8b-838c-bab1a23945af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:12.340551 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-04caab31-b1da-4d8b-838c-bab1a23945af None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:12.341737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-04caab31-b1da-4d8b-838c-bab1a23945af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:12.407920 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 502/998] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:12 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:48:12.524080 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2935393-3a5e-4114-b287-4cb5b57249dd None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:12.534926 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:12.628429 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2935393-3a5e-4114-b287-4cb5b57249dd None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:12.662213 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435a06f-b84d-4570-a201-17ade1836aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:12.662834 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435a06f-b84d-4570-a201-17ade1836aad None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:12.663111 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435a06f-b84d-4570-a201-17ade1836aad None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:12.663380 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435a06f-b84d-4570-a201-17ade1836aad None admin] PATH_INFO: `/v3/services/compute` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:12.663957 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5435a06f-b84d-4570-a201-17ade1836aad None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:12.665341 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5435a06f-b84d-4570-a201-17ade1836aad None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:12.673886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-5435a06f-b84d-4570-a201-17ade1836aad None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 20:48:12.673886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:12.673886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:48:12.673886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:48:12.673886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:48:12.673886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:48:12.673886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:48:12.673886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:48:12.673886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 20:48:12.673886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:12.673886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:48:12.673886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:12.673886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:12.673886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:12.673886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:12.673886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:12.673886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:12.674903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:12.674903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:12.674903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:12.674903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:12.674903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:12.674903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:12.674903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:48:12.674903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:48:12.674903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:48:12.674903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:48:12.674903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:12.674903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:12.674903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:12.674903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:12.674903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:12.674903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:12.675755 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:12.675755 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:12.675755 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:12.675755 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:12.675755 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:12.675755 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:12.675755 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:12.675755 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:12.675755 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:12.675755 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:12.675755 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:48:12.675755 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:48:12.675755 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 20:48:12.675755 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:12.675755 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 497/999] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:48:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:12.678181 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04caab31-b1da-4d8b-838c-bab1a23945af None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:12.680574 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435a06f-b84d-4570-a201-17ade1836aad None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:12.708124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:12.759473 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:12.783573 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04caab31-b1da-4d8b-838c-bab1a23945af None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:12.789164 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435a06f-b84d-4570-a201-17ade1836aad None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:12.822739 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0eaedf-2f2e-4154-8683-fd99d10576f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:12.823407 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0eaedf-2f2e-4154-8683-fd99d10576f7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:12.823690 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0eaedf-2f2e-4154-8683-fd99d10576f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:12.823964 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0eaedf-2f2e-4154-8683-fd99d10576f7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:12.824594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0eaedf-2f2e-4154-8683-fd99d10576f7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:12.825952 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0eaedf-2f2e-4154-8683-fd99d10576f7 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:12.826320 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4838e5-28b0-470a-9543-f01efd33f83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:12.826867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4838e5-28b0-470a-9543-f01efd33f83a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:12.827146 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4838e5-28b0-470a-9543-f01efd33f83a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:12.827418 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4838e5-28b0-470a-9543-f01efd33f83a None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:12.828045 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4838e5-28b0-470a-9543-f01efd33f83a None admin] RBAC: Adding query filter params (name=compute) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:12.828377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4838e5-28b0-470a-9543-f01efd33f83a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:12.829737 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4838e5-28b0-470a-9543-f01efd33f83a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:12.835353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 498/1000] 10.222.0.48 () {56 vars in 1112 bytes} [Fri Sep 9 20:48:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:48:12.835780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-1e0eaedf-2f2e-4154-8683-fd99d10576f7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 20:48:12.835780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:12.835780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:48:12.835780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:48:12.835780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:48:12.835780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:48:12.835780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:48:12.835780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:48:12.835780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 20:48:12.835780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:12.835780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:48:12.835780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:12.835780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:12.835780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:12.835780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:12.835780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:12.835780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:12.836910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:12.836910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:12.836910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:12.836910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:12.836910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:12.836910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:12.836910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:48:12.836910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:48:12.836910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:48:12.836910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:48:12.836910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:12.836910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:12.836910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:12.836910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:12.836910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:12.836910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:12.837920 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:12.837920 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:12.837920 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:12.837920 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:12.837920 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:12.837920 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:12.837920 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:12.837920 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:12.837920 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:12.837920 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:12.837920 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:48:12.837920 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:48:12.837920 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 20:48:12.837920 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:12.838927 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 503/1001] 10.222.0.48 () {56 vars in 1126 bytes} [Fri Sep 9 20:48:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:12.845349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4838e5-28b0-470a-9543-f01efd33f83a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:12.845532 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0eaedf-2f2e-4154-8683-fd99d10576f7 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:12.948656 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:12.952824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4838e5-28b0-470a-9543-f01efd33f83a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:12.959172 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0eaedf-2f2e-4154-8683-fd99d10576f7 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:12.993031 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e79a3f-958a-4efa-acdb-906837fb7a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:12.993653 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e79a3f-958a-4efa-acdb-906837fb7a40 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:12.993938 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e79a3f-958a-4efa-acdb-906837fb7a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:12.994213 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e79a3f-958a-4efa-acdb-906837fb7a40 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:12.994847 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e79a3f-958a-4efa-acdb-906837fb7a40 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:12.995154 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e79a3f-958a-4efa-acdb-906837fb7a40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:12.996569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e79a3f-958a-4efa-acdb-906837fb7a40 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:12.998370 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:13.001731 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 499/1002] 10.222.0.48 () {56 vars in 1122 bytes} [Fri Sep 9 20:48:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:13.003809 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e4ad1-5397-4f83-b9fc-564d079636e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:13.004475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e4ad1-5397-4f83-b9fc-564d079636e1 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:13.008936 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e4ad1-5397-4f83-b9fc-564d079636e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:13.008936 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e4ad1-5397-4f83-b9fc-564d079636e1 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:13.008936 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e4ad1-5397-4f83-b9fc-564d079636e1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:13.008936 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e4ad1-5397-4f83-b9fc-564d079636e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:13.008936 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e4ad1-5397-4f83-b9fc-564d079636e1 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:13.011813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e79a3f-958a-4efa-acdb-906837fb7a40 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:13.016013 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 504/1003] 10.222.0.48 () {56 vars in 1112 bytes} [Fri Sep 9 20:48:12 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:48:13.022267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e4ad1-5397-4f83-b9fc-564d079636e1 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:13.119130 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e79a3f-958a-4efa-acdb-906837fb7a40 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:13.137399 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e4ad1-5397-4f83-b9fc-564d079636e1 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:13.153486 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286383a-933a-409a-bdf4-b1e18aab442e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:13.154094 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286383a-933a-409a-bdf4-b1e18aab442e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:13.154372 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286383a-933a-409a-bdf4-b1e18aab442e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:13.154646 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286383a-933a-409a-bdf4-b1e18aab442e None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:13.155276 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5286383a-933a-409a-bdf4-b1e18aab442e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:13.155581 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5286383a-933a-409a-bdf4-b1e18aab442e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:13.157008 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5286383a-933a-409a-bdf4-b1e18aab442e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:13.166291 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 500/1004] 10.222.0.48 () {56 vars in 1122 bytes} [Fri Sep 9 20:48:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:13.167254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286383a-933a-409a-bdf4-b1e18aab442e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:13.181695 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989575d-0427-44d8-929e-d5890c3740a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:13.182417 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989575d-0427-44d8-929e-d5890c3740a9 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:13.182694 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989575d-0427-44d8-929e-d5890c3740a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:13.182966 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989575d-0427-44d8-929e-d5890c3740a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:13.183784 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b989575d-0427-44d8-929e-d5890c3740a9 None admin] RBAC: Adding query filter params (interface=public, service_id=a3159bace2274384bcc4a0431fc4717b, region_id=RegionOne) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:13.184141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b989575d-0427-44d8-929e-d5890c3740a9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:13.185491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b989575d-0427-44d8-929e-d5890c3740a9 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:13.192639 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 505/1005] 10.222.0.48 () {56 vars in 1251 bytes} [Fri Sep 9 20:48:13 2022] GET /identity/v3/endpoints?service_id=a3159bace2274384bcc4a0431fc4717b&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:48:13.198414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989575d-0427-44d8-929e-d5890c3740a9 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:13.219866 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:13.231551 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:13.270212 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286383a-933a-409a-bdf4-b1e18aab442e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:13.306110 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c4772-51c5-449d-9c82-d116985e3b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:13.306721 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c4772-51c5-449d-9c82-d116985e3b96 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:13.307003 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c4772-51c5-449d-9c82-d116985e3b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:13.307279 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c4772-51c5-449d-9c82-d116985e3b96 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:13.307950 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c4772-51c5-449d-9c82-d116985e3b96 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:13.308305 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989575d-0427-44d8-929e-d5890c3740a9 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:13.309377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c4772-51c5-449d-9c82-d116985e3b96 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:13.345315 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ffeeee-bdd0-4dc1-80fd-88ec3a1b59b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:13.345917 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffeeee-bdd0-4dc1-80fd-88ec3a1b59b2 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:13.346254 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffeeee-bdd0-4dc1-80fd-88ec3a1b59b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:13.346532 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffeeee-bdd0-4dc1-80fd-88ec3a1b59b2 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:13.347130 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ffeeee-bdd0-4dc1-80fd-88ec3a1b59b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:13.348509 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ffeeee-bdd0-4dc1-80fd-88ec3a1b59b2 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:13.354820 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 506/1006] 10.222.0.48 () {56 vars in 1087 bytes} [Fri Sep 9 20:48:13 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:48:13.366384 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 501/1007] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 20:48:13 2022] POST /identity/v3/endpoints => generated 325 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:48:13.893423 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72014c56-d821-4310-86f6-bf17c5d0cf8e None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:13.893785 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72014c56-d821-4310-86f6-bf17c5d0cf8e None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:13.893973 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72014c56-d821-4310-86f6-bf17c5d0cf8e None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:13.895558 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 507/1008] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:48:13.901222 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5eab6-d2df-4579-a800-d6249494ef50 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:13.901520 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5eab6-d2df-4579-a800-d6249494ef50 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:13.901799 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5eab6-d2df-4579-a800-d6249494ef50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:13.920572 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9fa5eab6-d2df-4579-a800-d6249494ef50 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:13.921762 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9fa5eab6-d2df-4579-a800-d6249494ef50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:14.000046 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 502/1009] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:13 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:48:14.273479 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72014c56-d821-4310-86f6-bf17c5d0cf8e None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:14.362150 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:14.378404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72014c56-d821-4310-86f6-bf17c5d0cf8e None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:14.411953 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28495d5-2fdd-4242-a239-5e450477cdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:14.412571 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28495d5-2fdd-4242-a239-5e450477cdd0 None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:14.412844 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28495d5-2fdd-4242-a239-5e450477cdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:14.413108 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28495d5-2fdd-4242-a239-5e450477cdd0 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:14.413754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28495d5-2fdd-4242-a239-5e450477cdd0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:14.415092 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28495d5-2fdd-4242-a239-5e450477cdd0 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:14.519187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 508/1010] 10.222.0.48 () {60 vars in 1141 bytes} [Fri Sep 9 20:48:14 2022] POST /identity/v3/services => generated 242 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:48:15.550093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebd71b-0c39-4035-a5dd-5eda5eb81010 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:15.550646 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebd71b-0c39-4035-a5dd-5eda5eb81010 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:15.550646 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebd71b-0c39-4035-a5dd-5eda5eb81010 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:15.552153 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 503/1011] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:48:15.557300 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2f94f-db56-4b16-b2cc-48ff6bff0b57 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:15.557694 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2f94f-db56-4b16-b2cc-48ff6bff0b57 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:15.557856 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2f94f-db56-4b16-b2cc-48ff6bff0b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:15.575914 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-83b2f94f-db56-4b16-b2cc-48ff6bff0b57 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:15.577130 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-83b2f94f-db56-4b16-b2cc-48ff6bff0b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:15.597921 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05267c-7c85-4774-aaf9-5d60452cf226 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:15.598197 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05267c-7c85-4774-aaf9-5d60452cf226 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:15.598472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05267c-7c85-4774-aaf9-5d60452cf226 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:15.600586 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 504/1012] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:48:15.605422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3673c-ed41-4748-945e-7da8a837d2b9 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:15.605708 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3673c-ed41-4748-945e-7da8a837d2b9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:15.605983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3673c-ed41-4748-945e-7da8a837d2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:15.624530 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-36a3673c-ed41-4748-945e-7da8a837d2b9 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:15.625776 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-36a3673c-ed41-4748-945e-7da8a837d2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:15.654542 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 509/1013] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:15 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:48:15.678027 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:15.699435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 505/1014] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:15 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:48:15.921012 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2f94f-db56-4b16-b2cc-48ff6bff0b57 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:15.968640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3673c-ed41-4748-945e-7da8a837d2b9 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:15.991799 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:16.026027 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2f94f-db56-4b16-b2cc-48ff6bff0b57 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:16.061040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e28ba-985f-4764-9d81-4342daf4a018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:16.061924 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e28ba-985f-4764-9d81-4342daf4a018 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:16.062205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e28ba-985f-4764-9d81-4342daf4a018 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:16.062480 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e28ba-985f-4764-9d81-4342daf4a018 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:16.063064 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e28ba-985f-4764-9d81-4342daf4a018 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:16.064466 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e28ba-985f-4764-9d81-4342daf4a018 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:16.074474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-762e28ba-985f-4764-9d81-4342daf4a018 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 20:48:16.074474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:16.074474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:48:16.074474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:48:16.074474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:48:16.074474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:48:16.074474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:48:16.074474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:48:16.074474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 20:48:16.074474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:16.074474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:48:16.074474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:16.074474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:16.074474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:16.074474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:16.074474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:16.074474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:16.075506 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:16.075506 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:16.075506 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:16.075506 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:16.075506 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:16.075506 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:16.075506 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:48:16.075506 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:48:16.075506 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:48:16.075506 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:48:16.075506 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:16.075506 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:16.075506 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:16.075506 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:16.075506 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:16.075506 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:16.076482 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:16.076482 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:16.076482 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:16.076482 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:16.076482 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:16.076482 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:16.076482 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:16.076482 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:16.076482 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:16.076482 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:16.076482 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:48:16.076482 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:48:16.076482 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 20:48:16.076482 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:16.076482 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 510/1015] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:48:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:16.081475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e28ba-985f-4764-9d81-4342daf4a018 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:16.090883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3673c-ed41-4748-945e-7da8a837d2b9 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:16.134598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585831f1-d4f3-4815-b6e2-7ae447f33782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:16.135239 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585831f1-d4f3-4815-b6e2-7ae447f33782 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:16.135523 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585831f1-d4f3-4815-b6e2-7ae447f33782 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:16.135791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585831f1-d4f3-4815-b6e2-7ae447f33782 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:16.136942 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585831f1-d4f3-4815-b6e2-7ae447f33782 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:16.138526 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585831f1-d4f3-4815-b6e2-7ae447f33782 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:16.140559 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 506/1016] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:48:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:48:16.145805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585831f1-d4f3-4815-b6e2-7ae447f33782 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:16.193990 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e28ba-985f-4764-9d81-4342daf4a018 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:16.200307 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:16.228671 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885bd665-c1c7-4027-b668-d5f8b1ca0d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:16.229279 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885bd665-c1c7-4027-b668-d5f8b1ca0d33 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:16.229554 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885bd665-c1c7-4027-b668-d5f8b1ca0d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:16.229823 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885bd665-c1c7-4027-b668-d5f8b1ca0d33 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:16.230446 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885bd665-c1c7-4027-b668-d5f8b1ca0d33 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:16.230746 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885bd665-c1c7-4027-b668-d5f8b1ca0d33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:16.232319 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885bd665-c1c7-4027-b668-d5f8b1ca0d33 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:16.233916 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:16.243980 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 511/1017] 10.222.0.48 () {56 vars in 1112 bytes} [Fri Sep 9 20:48:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:48:16.245536 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885bd665-c1c7-4027-b668-d5f8b1ca0d33 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:16.251599 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585831f1-d4f3-4815-b6e2-7ae447f33782 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:16.287372 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde8f39-4530-43ee-b3ce-a2f3f426c8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:16.287970 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde8f39-4530-43ee-b3ce-a2f3f426c8fd None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:16.288273 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde8f39-4530-43ee-b3ce-a2f3f426c8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:16.288538 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde8f39-4530-43ee-b3ce-a2f3f426c8fd None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:16.289176 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde8f39-4530-43ee-b3ce-a2f3f426c8fd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:16.290604 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde8f39-4530-43ee-b3ce-a2f3f426c8fd None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:16.349772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885bd665-c1c7-4027-b668-d5f8b1ca0d33 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:16.383568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520c448-02ff-46e4-93eb-44bd6c1eb5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:16.383568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520c448-02ff-46e4-93eb-44bd6c1eb5eb None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:16.383568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520c448-02ff-46e4-93eb-44bd6c1eb5eb None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:16.383568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520c448-02ff-46e4-93eb-44bd6c1eb5eb None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:16.386504 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520c448-02ff-46e4-93eb-44bd6c1eb5eb None admin] RBAC: Adding query filter params (type=compute) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:16.386504 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520c448-02ff-46e4-93eb-44bd6c1eb5eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:16.387445 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520c448-02ff-46e4-93eb-44bd6c1eb5eb None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:16.392429 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 512/1018] 10.222.0.48 () {56 vars in 1112 bytes} [Fri Sep 9 20:48:16 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:48:16.398541 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520c448-02ff-46e4-93eb-44bd6c1eb5eb None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:16.404753 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 507/1019] 10.222.0.48 () {60 vars in 1141 bytes} [Fri Sep 9 20:48:16 2022] POST /identity/v3/projects => generated 311 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:48:16.468095 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:16.512837 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520c448-02ff-46e4-93eb-44bd6c1eb5eb None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:16.548681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2aefbd-74d3-4cc7-bb59-7c4099a2ccc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:16.549301 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2aefbd-74d3-4cc7-bb59-7c4099a2ccc1 None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:16.549577 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2aefbd-74d3-4cc7-bb59-7c4099a2ccc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:16.549843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2aefbd-74d3-4cc7-bb59-7c4099a2ccc1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:16.550519 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2aefbd-74d3-4cc7-bb59-7c4099a2ccc1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:16.551892 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2aefbd-74d3-4cc7-bb59-7c4099a2ccc1 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:16.634667 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 513/1020] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 20:48:16 2022] POST /identity/v3/endpoints => generated 334 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:48:16.713368 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5e412-47c4-4f50-bd4c-9b81b8d05512 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:16.713951 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5e412-47c4-4f50-bd4c-9b81b8d05512 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:16.713951 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5e412-47c4-4f50-bd4c-9b81b8d05512 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:16.715293 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 508/1021] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:48:16.729183 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33bf124-5a64-4db6-97e7-039cf292b971 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:16.729183 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33bf124-5a64-4db6-97e7-039cf292b971 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:16.729183 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33bf124-5a64-4db6-97e7-039cf292b971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:16.749063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-c33bf124-5a64-4db6-97e7-039cf292b971 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:16.750295 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c33bf124-5a64-4db6-97e7-039cf292b971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:16.830412 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 514/1022] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:16 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:48:17.098701 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce5e412-47c4-4f50-bd4c-9b81b8d05512 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:17.199842 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:17.204512 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce5e412-47c4-4f50-bd4c-9b81b8d05512 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:17.239289 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d5e8e-ea8e-4837-8a67-ad50fa2cdf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:17.240352 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d5e8e-ea8e-4837-8a67-ad50fa2cdf38 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:17.240861 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d5e8e-ea8e-4837-8a67-ad50fa2cdf38 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:17.241336 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d5e8e-ea8e-4837-8a67-ad50fa2cdf38 None admin] PATH_INFO: `/v3/services/image` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:17.242178 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d5e8e-ea8e-4837-8a67-ad50fa2cdf38 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:17.243862 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d5e8e-ea8e-4837-8a67-ad50fa2cdf38 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:17.253764 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-6e7d5e8e-ea8e-4837-8a67-ad50fa2cdf38 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 20:48:17.253764 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:17.253764 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:48:17.253764 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:48:17.253764 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:48:17.253764 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:48:17.253764 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:48:17.253764 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:48:17.253764 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 20:48:17.253764 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:17.253764 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:48:17.253764 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:17.253764 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:17.253764 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:17.253764 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:17.253764 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:17.253764 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:17.256339 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:17.256339 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:17.256339 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:17.256339 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:17.256339 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:17.256339 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:17.256339 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:48:17.256339 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:48:17.256339 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:48:17.256339 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:48:17.256339 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:17.256339 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:17.256339 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:17.256339 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:17.256339 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:17.256339 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:17.257200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:17.257200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:17.257200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:17.257200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:17.257200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:17.257200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:17.257200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:17.257200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:17.257200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:17.257200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:17.257200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:48:17.257200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:48:17.257200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 20:48:17.257200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:17.257200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 509/1023] 10.222.0.48 () {56 vars in 1105 bytes} [Fri Sep 9 20:48:17 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:17.261726 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33bf124-5a64-4db6-97e7-039cf292b971 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:17.307197 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:17.362097 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33bf124-5a64-4db6-97e7-039cf292b971 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:17.397887 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a7fec0-a34f-494d-a9eb-deb09422cf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:17.398494 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7fec0-a34f-494d-a9eb-deb09422cf9d None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:17.398774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7fec0-a34f-494d-a9eb-deb09422cf9d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:17.399041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7fec0-a34f-494d-a9eb-deb09422cf9d None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:17.399671 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a7fec0-a34f-494d-a9eb-deb09422cf9d None admin] RBAC: Adding query filter params (name=image) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:17.399974 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a7fec0-a34f-494d-a9eb-deb09422cf9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:17.401362 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a7fec0-a34f-494d-a9eb-deb09422cf9d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:17.411118 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 515/1024] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:48:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:17.412348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d5e8e-ea8e-4837-8a67-ad50fa2cdf38 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:17.515360 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d5e8e-ea8e-4837-8a67-ad50fa2cdf38 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:17.548284 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c3c32-fca5-4da1-a2be-2769a263add4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:17.548888 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c3c32-fca5-4da1-a2be-2769a263add4 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:17.549170 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c3c32-fca5-4da1-a2be-2769a263add4 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:17.549444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c3c32-fca5-4da1-a2be-2769a263add4 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:17.550073 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c3c32-fca5-4da1-a2be-2769a263add4 None admin] RBAC: Adding query filter params (type=image) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:17.550380 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c3c32-fca5-4da1-a2be-2769a263add4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:17.551761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c3c32-fca5-4da1-a2be-2769a263add4 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:17.560389 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 510/1025] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:48:17 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:48:17.563360 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a7fec0-a34f-494d-a9eb-deb09422cf9d None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:17.667878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a7fec0-a34f-494d-a9eb-deb09422cf9d None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:17.677036 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:17.703222 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53dbf61-c6e8-4c4a-a3b0-4198754f7e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:17.703830 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53dbf61-c6e8-4c4a-a3b0-4198754f7e74 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:17.704137 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53dbf61-c6e8-4c4a-a3b0-4198754f7e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:17.704411 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53dbf61-c6e8-4c4a-a3b0-4198754f7e74 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:17.705107 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53dbf61-c6e8-4c4a-a3b0-4198754f7e74 None admin] RBAC: Adding query filter params (interface=public, service_id=c0832ff3c94c4046882a5e44d015a6d0, region_id=RegionOne) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:17.705415 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53dbf61-c6e8-4c4a-a3b0-4198754f7e74 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:17.706828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53dbf61-c6e8-4c4a-a3b0-4198754f7e74 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:17.717148 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 516/1026] 10.222.0.48 () {56 vars in 1251 bytes} [Fri Sep 9 20:48:17 2022] GET /identity/v3/endpoints?service_id=c0832ff3c94c4046882a5e44d015a6d0&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:48:17.718764 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c3c32-fca5-4da1-a2be-2769a263add4 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:17.748731 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:17.821405 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c3c32-fca5-4da1-a2be-2769a263add4 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:17.854324 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba0d58-4bcb-4dff-bdd1-b40870555cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:17.855049 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba0d58-4bcb-4dff-bdd1-b40870555cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:17.855341 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba0d58-4bcb-4dff-bdd1-b40870555cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:17.855617 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba0d58-4bcb-4dff-bdd1-b40870555cf7 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:17.856263 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba0d58-4bcb-4dff-bdd1-b40870555cf7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:17.857642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba0d58-4bcb-4dff-bdd1-b40870555cf7 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:17.868464 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 511/1027] 10.222.0.48 () {56 vars in 1087 bytes} [Fri Sep 9 20:48:17 2022] GET /identity/v3/services => generated 1957 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 20:48:18.653572 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ab719-f73c-4e4f-acfc-81afedac07b9 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:18.653852 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ab719-f73c-4e4f-acfc-81afedac07b9 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:18.654125 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ab719-f73c-4e4f-acfc-81afedac07b9 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:18.655605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 517/1028] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:48:18.660848 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2590d-2949-43e9-bef5-e85db8cb26ee None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:18.661133 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2590d-2949-43e9-bef5-e85db8cb26ee None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:18.661408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2590d-2949-43e9-bef5-e85db8cb26ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:18.679453 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-71b2590d-2949-43e9-bef5-e85db8cb26ee None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:18.680660 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-71b2590d-2949-43e9-bef5-e85db8cb26ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:18.750375 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 512/1029] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:48:18.919100 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8cad3-9cb3-41b5-bc4d-671c67a198a7 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:18.919608 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8cad3-9cb3-41b5-bc4d-671c67a198a7 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:18.919608 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8cad3-9cb3-41b5-bc4d-671c67a198a7 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:18.921032 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 518/1030] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:48:18.925918 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98255f7-6b86-4682-a2e4-769ad0d8d2c6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:18.926207 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98255f7-6b86-4682-a2e4-769ad0d8d2c6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:18.926480 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98255f7-6b86-4682-a2e4-769ad0d8d2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:18.945362 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c98255f7-6b86-4682-a2e4-769ad0d8d2c6 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:18.946533 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c98255f7-6b86-4682-a2e4-769ad0d8d2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:18.963924 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:19.016809 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 513/1031] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:48:19.021754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8cad3-9cb3-41b5-bc4d-671c67a198a7 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:19.129092 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8cad3-9cb3-41b5-bc4d-671c67a198a7 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:19.162568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a537e-e2f7-47f9-9129-0b5fda66c9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:19.163196 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a537e-e2f7-47f9-9129-0b5fda66c9be None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:19.163475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a537e-e2f7-47f9-9129-0b5fda66c9be None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:19.163745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a537e-e2f7-47f9-9129-0b5fda66c9be None admin] PATH_INFO: `/v3/domains/default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:19.164892 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a537e-e2f7-47f9-9129-0b5fda66c9be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:19.166311 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a537e-e2f7-47f9-9129-0b5fda66c9be None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:19.168044 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 519/1032] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:48:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:19.173586 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98255f7-6b86-4682-a2e4-769ad0d8d2c6 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:19.272571 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98255f7-6b86-4682-a2e4-769ad0d8d2c6 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:19.293776 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a537e-e2f7-47f9-9129-0b5fda66c9be None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:19.306314 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc0762b-b19f-4c6e-932c-3948204c2484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:19.306924 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0762b-b19f-4c6e-932c-3948204c2484 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:19.307205 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0762b-b19f-4c6e-932c-3948204c2484 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:19.307479 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0762b-b19f-4c6e-932c-3948204c2484 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:19.308166 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc0762b-b19f-4c6e-932c-3948204c2484 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:19.309598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc0762b-b19f-4c6e-932c-3948204c2484 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:19.336920 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:19.391878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a537e-e2f7-47f9-9129-0b5fda66c9be None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:19.405620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 514/1033] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Sep 9 20:48:19 2022] POST /identity/v3/users => generated 285 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:48:19.430643 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314f38a-ebef-4f09-9f4a-da74fa918897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:19.431670 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314f38a-ebef-4f09-9f4a-da74fa918897 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:19.432201 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314f38a-ebef-4f09-9f4a-da74fa918897 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:19.432618 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314f38a-ebef-4f09-9f4a-da74fa918897 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:19.445631 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f314f38a-ebef-4f09-9f4a-da74fa918897 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:48:19.450994 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-f314f38a-ebef-4f09-9f4a-da74fa918897 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 20:48:19.450994 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:19.450994 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:19.450994 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:19.450994 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:19.450994 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:19.450994 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:19.450994 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:19.450994 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:19.450994 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:19.450994 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:19.450994 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:19.450994 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:48:19.450994 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:48:19.450994 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:48:19.450994 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:48:19.452146 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:48:19.452146 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:48:19.452146 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:48:19.452146 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:48:19.452146 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:19.452146 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:19.452146 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:19.452146 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:19.452146 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:19.452146 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:19.452146 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:19.452146 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:19.452146 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:19.452146 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:19.452146 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:19.452146 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:19.453169 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:19.453169 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:19.453169 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:19.453169 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:19.453169 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:48:19.453169 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:48:19.453169 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:48:19.453169 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:48:19.453169 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:48:19.453169 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:48:19.453169 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 20:48:19.453169 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:19.455787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 520/1034] 10.222.0.48 () {56 vars in 1120 bytes} [Fri Sep 9 20:48:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:48:19.461766 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc0762b-b19f-4c6e-932c-3948204c2484 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:19.474672 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:19.574696 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc0762b-b19f-4c6e-932c-3948204c2484 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:19.611602 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f434c4-f94f-4356-b85d-1897acb6fa31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:19.612253 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f434c4-f94f-4356-b85d-1897acb6fa31 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:19.612537 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f434c4-f94f-4356-b85d-1897acb6fa31 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:19.612812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f434c4-f94f-4356-b85d-1897acb6fa31 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:19.613494 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f434c4-f94f-4356-b85d-1897acb6fa31 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:19.613805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f434c4-f94f-4356-b85d-1897acb6fa31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:19.615751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f434c4-f94f-4356-b85d-1897acb6fa31 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:19.624315 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 515/1035] 10.222.0.48 () {56 vars in 1115 bytes} [Fri Sep 9 20:48:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:48:19.630229 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314f38a-ebef-4f09-9f4a-da74fa918897 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:19.734419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:19.737984 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314f38a-ebef-4f09-9f4a-da74fa918897 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:19.773151 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b99113-74d5-4991-9830-41cdb04cafce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:19.773780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b99113-74d5-4991-9830-41cdb04cafce None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:19.774052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b99113-74d5-4991-9830-41cdb04cafce None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:19.774319 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b99113-74d5-4991-9830-41cdb04cafce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:19.784354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b99113-74d5-4991-9830-41cdb04cafce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:19.785784 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b99113-74d5-4991-9830-41cdb04cafce None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:19.800572 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-c3b99113-74d5-4991-9830-41cdb04cafce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:48:19.800572 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:19.800572 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:19.800572 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:19.800572 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:19.800572 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:19.800572 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:19.800572 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:19.800572 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:19.800572 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:19.800572 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:19.800572 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:19.800572 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:48:19.800572 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:48:19.800572 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:48:19.800572 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:48:19.801518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:19.801518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:19.801518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:19.801518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:19.801518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:19.801518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:19.801518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:19.801518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:19.801518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:19.801518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:19.801518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:19.801518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:19.801518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:19.801518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:19.801518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:19.801518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:19.801518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:48:19.802411 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:48:19.802411 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:48:19.802411 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:19.802411 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 521/1036] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:48:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:19.807348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f434c4-f94f-4356-b85d-1897acb6fa31 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:19.879000 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:19.909976 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f434c4-f94f-4356-b85d-1897acb6fa31 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:19.948200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8cd28-738c-487b-bef5-3ccd42be9462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:19.948815 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8cd28-738c-487b-bef5-3ccd42be9462 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:19.949104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8cd28-738c-487b-bef5-3ccd42be9462 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:19.949366 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8cd28-738c-487b-bef5-3ccd42be9462 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:19.950010 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8cd28-738c-487b-bef5-3ccd42be9462 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:19.950313 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8cd28-738c-487b-bef5-3ccd42be9462 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:19.951690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8cd28-738c-487b-bef5-3ccd42be9462 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:19.968908 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 516/1037] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:48:19.970082 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b99113-74d5-4991-9830-41cdb04cafce None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:20.033900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a92471-fb6f-4b3f-a4b2-a871957e3b76 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:20.034245 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a92471-fb6f-4b3f-a4b2-a871957e3b76 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:20.034463 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a92471-fb6f-4b3f-a4b2-a871957e3b76 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:20.035934 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 517/1038] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:48:20.041702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6c6d1-011f-4931-abec-cfac9727cbfc None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:20.041985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6c6d1-011f-4931-abec-cfac9727cbfc None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:20.042267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6c6d1-011f-4931-abec-cfac9727cbfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:20.061747 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-21b6c6d1-011f-4931-abec-cfac9727cbfc None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:20.063250 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-21b6c6d1-011f-4931-abec-cfac9727cbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:20.085332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b99113-74d5-4991-9830-41cdb04cafce None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:20.124414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d84e8f-eead-49ce-b34e-60476ce4da78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:20.125051 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d84e8f-eead-49ce-b34e-60476ce4da78 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:20.125334 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d84e8f-eead-49ce-b34e-60476ce4da78 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:20.125611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d84e8f-eead-49ce-b34e-60476ce4da78 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:20.134349 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d84e8f-eead-49ce-b34e-60476ce4da78 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:20.135784 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d84e8f-eead-49ce-b34e-60476ce4da78 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:20.141349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 518/1039] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:20 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:48:20.148147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-25d84e8f-eead-49ce-b34e-60476ce4da78 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:48:20.148147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:20.148147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:20.148147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:20.148147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:20.148147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:20.148147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:20.148147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:20.148147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:20.148147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:20.148147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:20.148147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:20.148147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:48:20.148147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:48:20.148147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:48:20.148147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:48:20.149071 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:20.149071 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:20.149071 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:48:20.149071 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:48:20.149071 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:48:20.149071 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:48:20.149071 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:20.149071 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:20.149071 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:20.149071 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:20.149071 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:20.149071 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:20.149071 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:20.149071 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:20.149071 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:20.149071 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:20.149071 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:20.149071 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:20.150011 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:20.150011 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:20.150011 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:48:20.150011 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:48:20.150011 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:48:20.150011 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:48:20.150011 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:48:20.150011 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:48:20.150011 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:48:20.150011 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:20.152351 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 522/1040] 10.222.0.48 () {56 vars in 1093 bytes} [Fri Sep 9 20:48:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:20.157331 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6c6d1-011f-4931-abec-cfac9727cbfc None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:20.217541 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:20.258488 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6c6d1-011f-4931-abec-cfac9727cbfc None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:20.292510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711ecfff-84f7-42b3-8e5e-fa0bfbddadee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:20.293115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711ecfff-84f7-42b3-8e5e-fa0bfbddadee None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:20.293404 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711ecfff-84f7-42b3-8e5e-fa0bfbddadee None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:20.293679 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711ecfff-84f7-42b3-8e5e-fa0bfbddadee None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:20.294407 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711ecfff-84f7-42b3-8e5e-fa0bfbddadee None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:20.294717 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711ecfff-84f7-42b3-8e5e-fa0bfbddadee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:20.296178 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711ecfff-84f7-42b3-8e5e-fa0bfbddadee None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:20.318354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 519/1041] 10.222.0.48 () {56 vars in 1133 bytes} [Fri Sep 9 20:48:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:20.319537 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d84e8f-eead-49ce-b34e-60476ce4da78 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:20.335403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:20.424491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d84e8f-eead-49ce-b34e-60476ce4da78 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:20.430767 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711ecfff-84f7-42b3-8e5e-fa0bfbddadee None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:20.466040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242a6b0c-60a6-4d5f-b886-1c7bc7f466de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:20.466654 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a6b0c-60a6-4d5f-b886-1c7bc7f466de None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:20.466937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a6b0c-60a6-4d5f-b886-1c7bc7f466de None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:20.467213 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a6b0c-60a6-4d5f-b886-1c7bc7f466de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:20.536765 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242a6b0c-60a6-4d5f-b886-1c7bc7f466de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:20.538215 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242a6b0c-60a6-4d5f-b886-1c7bc7f466de None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:20.543063 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711ecfff-84f7-42b3-8e5e-fa0bfbddadee None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:20.551990 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:20.562823 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:20.579455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07bdba-c555-4c07-8b2e-8fb8be8afb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:20.580148 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07bdba-c555-4c07-8b2e-8fb8be8afb94 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:20.580548 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07bdba-c555-4c07-8b2e-8fb8be8afb94 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:20.580829 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07bdba-c555-4c07-8b2e-8fb8be8afb94 None admin] PATH_INFO: `/v3/services/image` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:20.581450 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a07bdba-c555-4c07-8b2e-8fb8be8afb94 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:20.582796 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a07bdba-c555-4c07-8b2e-8fb8be8afb94 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:20.592317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-2a07bdba-c555-4c07-8b2e-8fb8be8afb94 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 20:48:20.592317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:20.592317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:48:20.592317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:48:20.592317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:48:20.592317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:48:20.592317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:48:20.592317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:48:20.592317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 20:48:20.592317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:20.592317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:48:20.592317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:20.592317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:20.592317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:20.592317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:20.592317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:20.592317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:20.593340 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:20.593340 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:20.593340 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:20.593340 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:20.593340 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:20.593340 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:20.593340 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:48:20.593340 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:48:20.593340 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:48:20.593340 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:48:20.593340 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:20.593340 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:20.593340 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:20.593340 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:20.593340 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:20.593340 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:20.594191 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:20.594191 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:20.594191 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:20.594191 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:20.594191 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:20.594191 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:20.594191 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:20.594191 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:20.594191 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:20.594191 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:20.594191 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:48:20.594191 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:48:20.594191 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 20:48:20.594191 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:20.594191 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 520/1042] 10.222.0.48 () {56 vars in 1105 bytes} [Fri Sep 9 20:48:20 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:48:20.599478 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07bdba-c555-4c07-8b2e-8fb8be8afb94 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:20.646729 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 523/1043] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:48:20 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:48:20.652335 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242a6b0c-60a6-4d5f-b886-1c7bc7f466de None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:20.704213 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07bdba-c555-4c07-8b2e-8fb8be8afb94 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:20.739324 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a752a13-da6a-44f1-9d7a-09dbda2cdd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:20.740622 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a752a13-da6a-44f1-9d7a-09dbda2cdd27 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:20.740902 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a752a13-da6a-44f1-9d7a-09dbda2cdd27 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:20.741169 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a752a13-da6a-44f1-9d7a-09dbda2cdd27 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:20.741794 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a752a13-da6a-44f1-9d7a-09dbda2cdd27 None admin] RBAC: Adding query filter params (name=image) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:20.742096 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a752a13-da6a-44f1-9d7a-09dbda2cdd27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:20.743577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a752a13-da6a-44f1-9d7a-09dbda2cdd27 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:20.752102 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 521/1044] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:48:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:48:20.754715 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a752a13-da6a-44f1-9d7a-09dbda2cdd27 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:20.758524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242a6b0c-60a6-4d5f-b886-1c7bc7f466de None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:20.773927 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:20.795236 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87c637-fa44-4f86-93b4-1817e7ee6df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:20.795891 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87c637-fa44-4f86-93b4-1817e7ee6df4 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:20.796201 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87c637-fa44-4f86-93b4-1817e7ee6df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:20.796479 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87c637-fa44-4f86-93b4-1817e7ee6df4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:20.807363 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df87c637-fa44-4f86-93b4-1817e7ee6df4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:20.807813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:20.808992 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df87c637-fa44-4f86-93b4-1817e7ee6df4 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:20.824840 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-df87c637-fa44-4f86-93b4-1817e7ee6df4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:48:20.824840 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:20.824840 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:20.824840 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:20.824840 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:20.824840 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:20.824840 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:20.824840 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:20.824840 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:20.824840 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:20.824840 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:20.824840 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:20.824840 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:48:20.824840 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:48:20.824840 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:48:20.824840 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:48:20.826272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:20.826272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:20.826272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:20.826272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:20.826272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:20.826272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:20.826272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:20.826272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:20.826272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:20.826272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:20.826272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:20.826272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:20.826272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:20.826272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:20.826272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:20.826272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:20.826272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:48:20.827361 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:48:20.827361 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:48:20.827361 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:20.830791 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 524/1045] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:48:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:48:20.831450 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87c637-fa44-4f86-93b4-1817e7ee6df4 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:20.866983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a752a13-da6a-44f1-9d7a-09dbda2cdd27 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:20.901311 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67e882-906f-441a-b6c2-470cb193df08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:20.901915 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67e882-906f-441a-b6c2-470cb193df08 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:20.902198 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67e882-906f-441a-b6c2-470cb193df08 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:20.902479 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67e882-906f-441a-b6c2-470cb193df08 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:20.903111 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b67e882-906f-441a-b6c2-470cb193df08 None admin] RBAC: Adding query filter params (type=image) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:20.903418 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b67e882-906f-441a-b6c2-470cb193df08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:20.904822 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b67e882-906f-441a-b6c2-470cb193df08 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:20.909697 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 522/1046] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:48:20 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:20.915758 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67e882-906f-441a-b6c2-470cb193df08 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:20.933498 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87c637-fa44-4f86-93b4-1817e7ee6df4 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:20.967987 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c1fb3e-7d20-490c-b6aa-20a4c8be2258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:20.968913 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1fb3e-7d20-490c-b6aa-20a4c8be2258 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:20.969358 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1fb3e-7d20-490c-b6aa-20a4c8be2258 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:20.969631 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1fb3e-7d20-490c-b6aa-20a4c8be2258 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:20.970377 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c1fb3e-7d20-490c-b6aa-20a4c8be2258 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:20.970914 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c1fb3e-7d20-490c-b6aa-20a4c8be2258 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:20.972002 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c1fb3e-7d20-490c-b6aa-20a4c8be2258 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:20.989158 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 525/1047] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:48:20.990468 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c1fb3e-7d20-490c-b6aa-20a4c8be2258 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:21.017551 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:21.028320 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67e882-906f-441a-b6c2-470cb193df08 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:21.039691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:21.063445 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a4e8e-3964-4612-bf7c-08da5792195e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:21.063987 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a4e8e-3964-4612-bf7c-08da5792195e None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:21.064295 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a4e8e-3964-4612-bf7c-08da5792195e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:21.064567 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a4e8e-3964-4612-bf7c-08da5792195e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:21.065266 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a4e8e-3964-4612-bf7c-08da5792195e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:21.066639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a4e8e-3964-4612-bf7c-08da5792195e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:21.097463 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c1fb3e-7d20-490c-b6aa-20a4c8be2258 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:21.135356 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae54b80-c2e3-433c-a23c-db476e20cdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:21.135989 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae54b80-c2e3-433c-a23c-db476e20cdbb None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:21.136471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae54b80-c2e3-433c-a23c-db476e20cdbb None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:21.136747 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae54b80-c2e3-433c-a23c-db476e20cdbb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:21.152342 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 523/1048] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 20:48:20 2022] POST /identity/v3/endpoints => generated 327 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:48:21.152556 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae54b80-c2e3-433c-a23c-db476e20cdbb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:21.152556 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae54b80-c2e3-433c-a23c-db476e20cdbb None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:21.157574 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-4ae54b80-c2e3-433c-a23c-db476e20cdbb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:48:21.157574 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:21.157574 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:21.157574 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:21.157574 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:21.157574 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:21.157574 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:21.157574 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:21.157574 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:21.157574 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:21.157574 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:21.157574 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:21.157574 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:48:21.157574 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:48:21.157574 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:48:21.158611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:48:21.158611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:21.158611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:21.158611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:21.158611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:21.158611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:21.158611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:21.158611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:21.158611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:21.158611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:21.158611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:21.158611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:21.158611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:21.158611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:21.158611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:21.158611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:21.158611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:21.159634 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:48:21.159634 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:48:21.159634 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:48:21.159634 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:48:21.159634 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:48:21.159634 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:48:21.159634 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:48:21.159634 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:21.164414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 526/1049] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:48:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:21.165744 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a4e8e-3964-4612-bf7c-08da5792195e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:21.288999 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a4e8e-3964-4612-bf7c-08da5792195e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:21.329275 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd23d3b-4654-4790-90d3-9c2c7a70590b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:21.329815 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd23d3b-4654-4790-90d3-9c2c7a70590b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:21.330095 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd23d3b-4654-4790-90d3-9c2c7a70590b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:21.330367 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd23d3b-4654-4790-90d3-9c2c7a70590b None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:21.331068 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd23d3b-4654-4790-90d3-9c2c7a70590b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:21.331375 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd23d3b-4654-4790-90d3-9c2c7a70590b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:21.334420 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd23d3b-4654-4790-90d3-9c2c7a70590b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:21.346718 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 524/1050] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:48:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:21.353269 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae54b80-c2e3-433c-a23c-db476e20cdbb None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:21.446993 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:21.463393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae54b80-c2e3-433c-a23c-db476e20cdbb None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:21.497605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea008717-3b5e-4b94-9d46-80f8e2009af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:21.498291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea008717-3b5e-4b94-9d46-80f8e2009af7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:21.498577 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea008717-3b5e-4b94-9d46-80f8e2009af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:21.498848 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea008717-3b5e-4b94-9d46-80f8e2009af7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:21.499572 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea008717-3b5e-4b94-9d46-80f8e2009af7 None admin] RBAC: Adding query filter params (role.id=63740f83debd49b6a81f640fe2a07ce2, scope.project.id=91319aa782334fb9930770029b74e84a, user.id=613b845a9b4d432daf8d478fc3970708) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:21.499881 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea008717-3b5e-4b94-9d46-80f8e2009af7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:21.501338 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea008717-3b5e-4b94-9d46-80f8e2009af7 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:21.508095 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 527/1051] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:48:21 2022] GET /identity/v3/role_assignments?user.id=613b845a9b4d432daf8d478fc3970708&scope.project.id=91319aa782334fb9930770029b74e84a&role.id=63740f83debd49b6a81f640fe2a07ce2 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:48:21.646536 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786abd6-b232-43e6-8dc6-a851c5c86598 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:21.646851 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786abd6-b232-43e6-8dc6-a851c5c86598 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:21.647017 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786abd6-b232-43e6-8dc6-a851c5c86598 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:21.648486 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 525/1052] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:48:21.654328 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f57189-ccdb-4727-aa67-1a44ad17b296 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:21.654659 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f57189-ccdb-4727-aa67-1a44ad17b296 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:21.654921 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f57189-ccdb-4727-aa67-1a44ad17b296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:21.675297 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-d2f57189-ccdb-4727-aa67-1a44ad17b296 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:21.676548 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d2f57189-ccdb-4727-aa67-1a44ad17b296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:21.743651 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 528/1053] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:48:22.013698 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6786abd6-b232-43e6-8dc6-a851c5c86598 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:22.038752 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:22.119987 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6786abd6-b232-43e6-8dc6-a851c5c86598 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:22.160448 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c42af-5466-4ae3-9f57-3db986964a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:22.161086 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c42af-5466-4ae3-9f57-3db986964a34 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:22.161385 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c42af-5466-4ae3-9f57-3db986964a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:22.161675 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c42af-5466-4ae3-9f57-3db986964a34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:22.171969 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9c42af-5466-4ae3-9f57-3db986964a34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:48:22.176791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-df9c42af-5466-4ae3-9f57-3db986964a34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:48:22.176791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:22.176791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:22.176791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:22.176791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:22.176791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:22.176791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:22.176791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:22.176791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:22.176791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:22.176791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:22.176791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:22.176791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:48:22.176791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:48:22.176791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:48:22.176791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:48:22.178226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:48:22.178226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:48:22.178226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:48:22.178226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:48:22.178226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:22.178226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:22.178226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:22.178226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:22.178226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:22.178226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:22.178226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:22.178226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:22.178226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:22.178226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:22.178226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:22.178226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:22.179088 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:22.179088 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:22.179088 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:22.179088 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:22.179088 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:48:22.179088 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:48:22.179088 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:48:22.179088 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:48:22.179088 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:48:22.179088 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:48:22.179088 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:48:22.179088 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:22.179088 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 526/1054] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:48:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:22.185737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f57189-ccdb-4727-aa67-1a44ad17b296 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:22.266069 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:22.287453 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f57189-ccdb-4727-aa67-1a44ad17b296 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:22.323752 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c09def-8f7d-441f-b546-7d02469f5675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:22.324604 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c09def-8f7d-441f-b546-7d02469f5675 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:22.325031 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c09def-8f7d-441f-b546-7d02469f5675 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:22.325448 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c09def-8f7d-441f-b546-7d02469f5675 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:22.326271 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c09def-8f7d-441f-b546-7d02469f5675 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:22.326721 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c09def-8f7d-441f-b546-7d02469f5675 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:22.328262 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c09def-8f7d-441f-b546-7d02469f5675 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:22.341265 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 529/1055] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:48:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:22.342550 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c42af-5466-4ae3-9f57-3db986964a34 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:22.433920 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:22.452288 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c42af-5466-4ae3-9f57-3db986964a34 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:22.486376 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274fe18-16fd-46bc-9ce8-3fb6e5d1e0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:22.487065 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274fe18-16fd-46bc-9ce8-3fb6e5d1e0df None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:22.487353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274fe18-16fd-46bc-9ce8-3fb6e5d1e0df None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:22.487629 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274fe18-16fd-46bc-9ce8-3fb6e5d1e0df None admin] PATH_INFO: `/v3/users/b2f506f005d14e6aaee55b470fc661ee` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:22.501461 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4274fe18-16fd-46bc-9ce8-3fb6e5d1e0df None admin] RBAC: Authorizing `identity:get_user(user_id=b2f506f005d14e6aaee55b470fc661ee)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:22.502900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4274fe18-16fd-46bc-9ce8-3fb6e5d1e0df None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:22.509324 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 527/1056] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:48:22 2022] GET /identity/v3/users/b2f506f005d14e6aaee55b470fc661ee => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:22.510275 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c09def-8f7d-441f-b546-7d02469f5675 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:22.609133 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c09def-8f7d-441f-b546-7d02469f5675 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:22.642038 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2661fd4e-9eaa-4b26-af07-103e4e785455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:22.642636 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661fd4e-9eaa-4b26-af07-103e4e785455 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:22.642919 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661fd4e-9eaa-4b26-af07-103e4e785455 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:22.643193 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661fd4e-9eaa-4b26-af07-103e4e785455 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:22.653605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:22.711508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2661fd4e-9eaa-4b26-af07-103e4e785455 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:22.712987 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2661fd4e-9eaa-4b26-af07-103e4e785455 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:22.813067 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 530/1057] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:48:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 20:48:22.818723 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274fe18-16fd-46bc-9ce8-3fb6e5d1e0df None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:22.918439 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274fe18-16fd-46bc-9ce8-3fb6e5d1e0df None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:22.951715 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3c917-68a5-419a-a53a-dd5787ab83d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:22.952435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3c917-68a5-419a-a53a-dd5787ab83d6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:22.952714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3c917-68a5-419a-a53a-dd5787ab83d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:22.952984 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3c917-68a5-419a-a53a-dd5787ab83d6 None admin] PATH_INFO: `/v3/projects/8a5d3e15d53443178423add213e268d0` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:22.954097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3c917-68a5-419a-a53a-dd5787ab83d6 None admin] RBAC: Authorizing `identity:get_project(project_id=8a5d3e15d53443178423add213e268d0)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:22.955520 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3c917-68a5-419a-a53a-dd5787ab83d6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:22.962850 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 528/1058] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:48:22 2022] GET /identity/v3/projects/8a5d3e15d53443178423add213e268d0 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:22.964058 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2661fd4e-9eaa-4b26-af07-103e4e785455 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:23.012958 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:23.067382 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2661fd4e-9eaa-4b26-af07-103e4e785455 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:23.103799 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6556c509-308e-4ac2-a16e-8640b3c1693e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:23.104437 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556c509-308e-4ac2-a16e-8640b3c1693e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:23.104717 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556c509-308e-4ac2-a16e-8640b3c1693e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:23.104987 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556c509-308e-4ac2-a16e-8640b3c1693e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:23.105769 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6556c509-308e-4ac2-a16e-8640b3c1693e None admin] RBAC: Adding query filter params (role.id=fcbf519fbc84458f913671a5a1d0914c, scope.project.id=8a5d3e15d53443178423add213e268d0, user.id=b2f506f005d14e6aaee55b470fc661ee) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:23.106076 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6556c509-308e-4ac2-a16e-8640b3c1693e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:23.107541 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6556c509-308e-4ac2-a16e-8640b3c1693e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:23.113367 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 531/1059] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:48:22 2022] GET /identity/v3/role_assignments?user.id=b2f506f005d14e6aaee55b470fc661ee&scope.project.id=8a5d3e15d53443178423add213e268d0&role.id=fcbf519fbc84458f913671a5a1d0914c => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:23.339631 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41801332-9ccf-49f8-8cbe-a379669110fa None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:23.340167 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41801332-9ccf-49f8-8cbe-a379669110fa None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:23.340167 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41801332-9ccf-49f8-8cbe-a379669110fa None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:23.341593 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 529/1060] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:48:23.346717 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea553009-ef01-42ff-8ae3-81cb86dc11b4 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:23.347108 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea553009-ef01-42ff-8ae3-81cb86dc11b4 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:23.347291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea553009-ef01-42ff-8ae3-81cb86dc11b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:23.365288 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-ea553009-ef01-42ff-8ae3-81cb86dc11b4 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:23.366458 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ea553009-ef01-42ff-8ae3-81cb86dc11b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:23.435420 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 532/1061] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:48:23.656904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e221e-7864-42e0-8c08-5abc2b898bfa None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:23.657231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e221e-7864-42e0-8c08-5abc2b898bfa None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:23.657459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e221e-7864-42e0-8c08-5abc2b898bfa None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:23.658923 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 530/1062] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:48:23.663888 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31198737-e279-4044-bae2-bb2f77b53cd7 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:23.664213 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31198737-e279-4044-bae2-bb2f77b53cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:23.664490 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31198737-e279-4044-bae2-bb2f77b53cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:23.682781 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-31198737-e279-4044-bae2-bb2f77b53cd7 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:23.683958 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-31198737-e279-4044-bae2-bb2f77b53cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:23.710682 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:23.714429 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e221e-7864-42e0-8c08-5abc2b898bfa None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:23.725122 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:23.756104 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 533/1063] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:48:23.823815 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e221e-7864-42e0-8c08-5abc2b898bfa None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:23.857012 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e9270-a1f8-4a92-8398-d53170e39c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:23.857622 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e9270-a1f8-4a92-8398-d53170e39c17 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:23.857898 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e9270-a1f8-4a92-8398-d53170e39c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:23.858167 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e9270-a1f8-4a92-8398-d53170e39c17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:23.924656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e9270-a1f8-4a92-8398-d53170e39c17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:23.926023 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e9270-a1f8-4a92-8398-d53170e39c17 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:23.935941 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:24.030713 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31198737-e279-4044-bae2-bb2f77b53cd7 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:24.036121 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 531/1064] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:48:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 20:48:24.038339 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e9270-a1f8-4a92-8398-d53170e39c17 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:24.138040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31198737-e279-4044-bae2-bb2f77b53cd7 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:24.140570 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e9270-a1f8-4a92-8398-d53170e39c17 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:24.152192 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:24.181161 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cbaf4-d45d-4958-b5e1-819a8a33f12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:24.181713 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cbaf4-d45d-4958-b5e1-819a8a33f12e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:24.181988 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cbaf4-d45d-4958-b5e1-819a8a33f12e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:24.182261 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cbaf4-d45d-4958-b5e1-819a8a33f12e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:24.183660 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb7fac-dfec-4ad7-b3b1-07acf8edeb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:24.184325 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb7fac-dfec-4ad7-b3b1-07acf8edeb56 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:24.184604 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb7fac-dfec-4ad7-b3b1-07acf8edeb56 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:24.184874 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb7fac-dfec-4ad7-b3b1-07acf8edeb56 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:24.192513 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415cbaf4-d45d-4958-b5e1-819a8a33f12e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:24.194479 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415cbaf4-d45d-4958-b5e1-819a8a33f12e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:24.196217 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eb7fac-dfec-4ad7-b3b1-07acf8edeb56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:48:24.201040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-f4eb7fac-dfec-4ad7-b3b1-07acf8edeb56 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 20:48:24.201040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:24.201040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:24.201040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:24.201040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:24.201040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:24.201040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:24.201040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:24.201040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:24.201040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:24.201040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:24.201040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:24.201040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:48:24.201040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:48:24.201040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:48:24.201040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:48:24.201982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:48:24.201982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:48:24.201982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:48:24.201982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:48:24.201982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:24.201982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:24.201982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:24.201982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:24.201982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:24.201982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:24.201982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:24.201982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:24.201982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:24.201982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:24.201982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:24.201982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:24.202845 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:24.202845 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:24.202845 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:24.202845 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:24.202845 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:48:24.202845 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:48:24.202845 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:48:24.202845 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:48:24.202845 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:48:24.202845 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:48:24.202845 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 20:48:24.202845 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:24.207183 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 534/1065] 10.222.0.48 () {56 vars in 1120 bytes} [Fri Sep 9 20:48:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:48:24.207893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb7fac-dfec-4ad7-b3b1-07acf8edeb56 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:24.211137 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-415cbaf4-d45d-4958-b5e1-819a8a33f12e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:48:24.211137 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:24.211137 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:24.211137 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:24.211137 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:24.211137 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:24.211137 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:24.211137 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:24.211137 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:24.211137 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:24.211137 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:24.211137 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:24.211137 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:48:24.211137 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:48:24.211137 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:48:24.211137 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:48:24.212081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:24.212081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:24.212081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:24.212081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:24.212081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:24.212081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:24.212081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:24.212081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:24.212081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:24.212081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:24.212081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:24.212081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:24.212081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:24.212081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:24.212081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:24.212081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:24.212081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:48:24.213002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:48:24.213002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:48:24.213002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:24.213002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 532/1066] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:48:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:48:24.218868 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cbaf4-d45d-4958-b5e1-819a8a33f12e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:24.235639 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:24.318603 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb7fac-dfec-4ad7-b3b1-07acf8edeb56 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:24.327918 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cbaf4-d45d-4958-b5e1-819a8a33f12e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:24.355123 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4f483-134c-4747-a801-a181d868b2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:24.355772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4f483-134c-4747-a801-a181d868b2c8 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:24.356011 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4f483-134c-4747-a801-a181d868b2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:24.356304 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4f483-134c-4747-a801-a181d868b2c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:24.356971 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d4f483-134c-4747-a801-a181d868b2c8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:24.357275 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d4f483-134c-4747-a801-a181d868b2c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:24.358642 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d4f483-134c-4747-a801-a181d868b2c8 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:24.363426 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57c043-2b50-4630-a19d-9eaf28f7ef45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:24.365121 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57c043-2b50-4630-a19d-9eaf28f7ef45 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:24.365547 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57c043-2b50-4630-a19d-9eaf28f7ef45 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:24.365965 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57c043-2b50-4630-a19d-9eaf28f7ef45 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:24.366741 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57c043-2b50-4630-a19d-9eaf28f7ef45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:24.367519 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 535/1067] 10.222.0.48 () {56 vars in 1115 bytes} [Fri Sep 9 20:48:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:24.367700 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57c043-2b50-4630-a19d-9eaf28f7ef45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:24.369122 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57c043-2b50-4630-a19d-9eaf28f7ef45 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:24.373008 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4f483-134c-4747-a801-a181d868b2c8 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:24.392164 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 533/1068] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:48:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:24.463853 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:24.479002 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4f483-134c-4747-a801-a181d868b2c8 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:24.516557 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aae8ab-5381-4467-9f32-2330bdb6c3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:24.517210 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aae8ab-5381-4467-9f32-2330bdb6c3b6 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:24.517493 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aae8ab-5381-4467-9f32-2330bdb6c3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:24.517766 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aae8ab-5381-4467-9f32-2330bdb6c3b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:24.528811 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aae8ab-5381-4467-9f32-2330bdb6c3b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:24.530276 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aae8ab-5381-4467-9f32-2330bdb6c3b6 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:24.545961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-d5aae8ab-5381-4467-9f32-2330bdb6c3b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:48:24.545961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:24.545961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:24.545961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:24.545961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:24.545961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:24.545961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:24.545961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:24.545961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:24.545961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:24.545961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:24.545961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:24.545961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:48:24.545961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:48:24.545961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:48:24.545961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:48:24.547033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:24.547033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:24.547033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:24.547033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:24.547033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:24.547033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:24.547033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:24.547033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:24.547033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:24.547033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:24.547033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:24.547033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:24.547033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:24.547033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:24.547033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:24.547033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:24.547033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:48:24.547935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:48:24.547935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:48:24.547935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:24.552469 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 536/1069] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:48:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:48:24.553581 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57c043-2b50-4630-a19d-9eaf28f7ef45 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:24.577682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:24.659766 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57c043-2b50-4630-a19d-9eaf28f7ef45 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:24.697357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63dd982-a056-4994-aed6-2ac9c099c696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:24.697973 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63dd982-a056-4994-aed6-2ac9c099c696 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:24.698244 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63dd982-a056-4994-aed6-2ac9c099c696 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:24.698522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63dd982-a056-4994-aed6-2ac9c099c696 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:24.699174 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63dd982-a056-4994-aed6-2ac9c099c696 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:24.699478 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63dd982-a056-4994-aed6-2ac9c099c696 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:24.700917 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63dd982-a056-4994-aed6-2ac9c099c696 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:24.712450 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 534/1070] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:48:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:24.718614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aae8ab-5381-4467-9f32-2330bdb6c3b6 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:24.818715 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aae8ab-5381-4467-9f32-2330bdb6c3b6 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:24.851076 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265dd77b-b692-47df-bb2b-03e2fa23d744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:24.851696 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265dd77b-b692-47df-bb2b-03e2fa23d744 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:24.851970 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265dd77b-b692-47df-bb2b-03e2fa23d744 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:24.852269 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265dd77b-b692-47df-bb2b-03e2fa23d744 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:24.859531 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265dd77b-b692-47df-bb2b-03e2fa23d744 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:24.860980 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265dd77b-b692-47df-bb2b-03e2fa23d744 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:24.862643 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:24.872949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-265dd77b-b692-47df-bb2b-03e2fa23d744 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:48:24.872949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:24.872949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:24.872949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:24.872949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:24.872949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:24.872949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:24.872949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:24.872949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:24.872949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:24.872949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:24.872949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:24.872949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:48:24.872949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:48:24.872949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:48:24.872949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:48:24.881923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:24.881923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:24.881923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:48:24.881923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:48:24.881923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:48:24.881923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:48:24.881923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:24.881923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:24.881923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:24.881923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:24.881923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:24.881923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:24.881923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:24.881923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:24.881923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:24.881923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:24.881923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:24.881923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:24.883064 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:24.883064 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:24.883064 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:48:24.883064 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:48:24.883064 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:48:24.883064 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:48:24.883064 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:48:24.883064 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:48:24.883064 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:48:24.883064 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:24.883064 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 537/1071] 10.222.0.48 () {56 vars in 1093 bytes} [Fri Sep 9 20:48:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:24.883759 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63dd982-a056-4994-aed6-2ac9c099c696 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:24.959060 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:24.977719 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63dd982-a056-4994-aed6-2ac9c099c696 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:25.013391 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ef8e8-30d4-46e0-b21e-e24c8993597a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:25.014011 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ef8e8-30d4-46e0-b21e-e24c8993597a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:25.014291 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ef8e8-30d4-46e0-b21e-e24c8993597a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:25.014562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ef8e8-30d4-46e0-b21e-e24c8993597a None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:25.015290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9ef8e8-30d4-46e0-b21e-e24c8993597a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:25.015594 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9ef8e8-30d4-46e0-b21e-e24c8993597a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:25.017056 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9ef8e8-30d4-46e0-b21e-e24c8993597a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:25.034768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 535/1072] 10.222.0.48 () {56 vars in 1133 bytes} [Fri Sep 9 20:48:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:25.041139 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265dd77b-b692-47df-bb2b-03e2fa23d744 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:25.138052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265dd77b-b692-47df-bb2b-03e2fa23d744 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:25.173589 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f0f861-c7a7-41b2-b30f-853e3364e86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:25.174254 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f0f861-c7a7-41b2-b30f-853e3364e86e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:25.174535 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f0f861-c7a7-41b2-b30f-853e3364e86e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:25.174806 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f0f861-c7a7-41b2-b30f-853e3364e86e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:25.186750 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f0f861-c7a7-41b2-b30f-853e3364e86e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:25.188243 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f0f861-c7a7-41b2-b30f-853e3364e86e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:25.202332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-d1f0f861-c7a7-41b2-b30f-853e3364e86e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:48:25.202332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:25.202332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:25.202332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:25.202332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:25.202332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:25.202332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:25.202332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:25.202332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:25.202332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:25.202332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:25.202332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:25.202332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:48:25.202332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:48:25.202332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:48:25.202332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:48:25.203354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:25.203354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:25.203354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:25.203354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:25.203354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:25.203354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:25.203354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:25.203354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:25.203354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:25.203354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:25.203354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:25.203354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:25.203354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:25.203354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:25.203354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:25.203354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:25.203354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:48:25.204300 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:48:25.204300 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:48:25.204300 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:25.204300 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 538/1073] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:48:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:25.209601 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ef8e8-30d4-46e0-b21e-e24c8993597a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:25.273480 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a8ec5-bb35-493c-91d1-9c3ad42e87c8 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:25.273694 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a8ec5-bb35-493c-91d1-9c3ad42e87c8 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:25.273973 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a8ec5-bb35-493c-91d1-9c3ad42e87c8 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:25.275298 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 539/1074] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:48:25.280179 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f59d57-6a88-4ced-988c-a93471730116 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:25.280467 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f59d57-6a88-4ced-988c-a93471730116 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:25.280750 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f59d57-6a88-4ced-988c-a93471730116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:25.298934 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-c9f59d57-6a88-4ced-988c-a93471730116 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:25.300150 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c9f59d57-6a88-4ced-988c-a93471730116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:25.314037 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ef8e8-30d4-46e0-b21e-e24c8993597a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:25.345725 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:25.349738 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c95c2-f27d-41c4-8f39-96485e4871b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:25.350342 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c95c2-f27d-41c4-8f39-96485e4871b1 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:25.350622 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c95c2-f27d-41c4-8f39-96485e4871b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:25.350903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c95c2-f27d-41c4-8f39-96485e4871b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:25.351548 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c95c2-f27d-41c4-8f39-96485e4871b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:25.351855 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c95c2-f27d-41c4-8f39-96485e4871b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:25.353274 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c95c2-f27d-41c4-8f39-96485e4871b1 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:25.368140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:25.368140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 536/1075] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:48:25.368801 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 540/1076] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:48:25.370429 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c95c2-f27d-41c4-8f39-96485e4871b1 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:25.473216 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c95c2-f27d-41c4-8f39-96485e4871b1 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:25.505520 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5124db75-682b-404f-8acf-da435031d964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:25.506150 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124db75-682b-404f-8acf-da435031d964 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:25.506427 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124db75-682b-404f-8acf-da435031d964 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:25.506694 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124db75-682b-404f-8acf-da435031d964 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:25.513104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5124db75-682b-404f-8acf-da435031d964 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:25.514536 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5124db75-682b-404f-8acf-da435031d964 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:25.526115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-5124db75-682b-404f-8acf-da435031d964 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:48:25.526115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:25.526115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:25.526115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:25.526115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:25.526115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:25.526115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:25.526115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:25.526115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:25.526115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:25.526115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:25.526115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:25.526115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:48:25.526115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:48:25.526115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:48:25.526967 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:48:25.526967 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:25.526967 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:25.526967 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:25.526967 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:25.526967 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:25.526967 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:25.526967 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:25.526967 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:25.526967 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:25.526967 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:25.526967 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:25.526967 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:25.526967 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:25.526967 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:25.526967 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:25.526967 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:25.527868 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:48:25.527868 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:48:25.527868 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:48:25.527868 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:48:25.527868 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:48:25.527868 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:48:25.527868 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:48:25.527868 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:25.527868 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 537/1077] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:48:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:25.532976 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f59d57-6a88-4ced-988c-a93471730116 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:25.627297 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f59d57-6a88-4ced-988c-a93471730116 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:25.642566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5124db75-682b-404f-8acf-da435031d964 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:25.660713 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2358d-c952-4a73-aa61-de9d62db3180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:25.661328 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2358d-c952-4a73-aa61-de9d62db3180 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:25.661607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2358d-c952-4a73-aa61-de9d62db3180 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:25.661876 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2358d-c952-4a73-aa61-de9d62db3180 None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:25.662569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c2358d-c952-4a73-aa61-de9d62db3180 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:25.662873 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c2358d-c952-4a73-aa61-de9d62db3180 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:25.664333 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c2358d-c952-4a73-aa61-de9d62db3180 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:25.675499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 541/1078] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:48:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:25.681267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2358d-c952-4a73-aa61-de9d62db3180 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:25.704158 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:25.722298 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:25.745622 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5124db75-682b-404f-8acf-da435031d964 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:25.784741 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2358d-c952-4a73-aa61-de9d62db3180 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:25.785515 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac1e7d7-e4d8-44c2-b88b-0447d9e913e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:25.786148 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac1e7d7-e4d8-44c2-b88b-0447d9e913e9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:25.786451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac1e7d7-e4d8-44c2-b88b-0447d9e913e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:25.786724 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac1e7d7-e4d8-44c2-b88b-0447d9e913e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:25.796600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac1e7d7-e4d8-44c2-b88b-0447d9e913e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:48:25.801782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-5ac1e7d7-e4d8-44c2-b88b-0447d9e913e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:48:25.801782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:25.801782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:25.801782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:25.801782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:25.801782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:25.801782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:25.801782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:25.801782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:25.801782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:25.801782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:25.801782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:25.801782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:48:25.801782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:48:25.801782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:48:25.801782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:48:25.803250 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:48:25.803250 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:48:25.803250 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:48:25.803250 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:48:25.803250 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:25.803250 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:25.803250 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:25.803250 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:25.803250 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:25.803250 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:25.803250 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:25.803250 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:25.803250 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:25.803250 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:25.803250 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:25.803250 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:25.804147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:25.804147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:25.804147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:25.804147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:25.804147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:48:25.804147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:48:25.804147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:48:25.804147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:48:25.804147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:48:25.804147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:48:25.804147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:48:25.804147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:25.804147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 538/1079] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:48:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:48:25.809282 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac1e7d7-e4d8-44c2-b88b-0447d9e913e9 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:25.819669 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d472b-59a9-4042-a2b7-dd0de6d4c345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:25.820371 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d472b-59a9-4042-a2b7-dd0de6d4c345 None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:25.820653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d472b-59a9-4042-a2b7-dd0de6d4c345 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:25.820928 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d472b-59a9-4042-a2b7-dd0de6d4c345 None admin] PATH_INFO: `/v3/projects/91319aa782334fb9930770029b74e84a/users/613b845a9b4d432daf8d478fc3970708/roles/63740f83debd49b6a81f640fe2a07ce2` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:25.823095 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3d472b-59a9-4042-a2b7-dd0de6d4c345 None admin] RBAC: Authorizing `identity:create_grant(project_id=91319aa782334fb9930770029b74e84a, user_id=613b845a9b4d432daf8d478fc3970708, role_id=63740f83debd49b6a81f640fe2a07ce2)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:25.824569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3d472b-59a9-4042-a2b7-dd0de6d4c345 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:25.878466 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 542/1080] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Sep 9 20:48:25 2022] PUT /identity/v3/projects/91319aa782334fb9930770029b74e84a/users/613b845a9b4d432daf8d478fc3970708/roles/63740f83debd49b6a81f640fe2a07ce2 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 20:48:25.908819 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac1e7d7-e4d8-44c2-b88b-0447d9e913e9 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:25.926849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:25.946451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ead81-6226-44c5-ae6e-4871d8aabd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:25.947061 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ead81-6226-44c5-ae6e-4871d8aabd67 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:25.947372 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ead81-6226-44c5-ae6e-4871d8aabd67 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:25.947643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ead81-6226-44c5-ae6e-4871d8aabd67 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:25.948366 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63ead81-6226-44c5-ae6e-4871d8aabd67 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:25.948668 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63ead81-6226-44c5-ae6e-4871d8aabd67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:25.950077 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63ead81-6226-44c5-ae6e-4871d8aabd67 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:25.964985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 539/1081] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:48:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:48:25.966483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d472b-59a9-4042-a2b7-dd0de6d4c345 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:26.072905 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:26.075756 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d472b-59a9-4042-a2b7-dd0de6d4c345 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:26.112429 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a108e-c045-4544-9f55-8035164cefd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:26.113091 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a108e-c045-4544-9f55-8035164cefd5 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:26.113378 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a108e-c045-4544-9f55-8035164cefd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:26.113639 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a108e-c045-4544-9f55-8035164cefd5 None admin] PATH_INFO: `/v3/users/b2f506f005d14e6aaee55b470fc661ee` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:26.115381 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a108e-c045-4544-9f55-8035164cefd5 None admin] RBAC: Authorizing `identity:get_user(user_id=b2f506f005d14e6aaee55b470fc661ee)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:26.116857 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a108e-c045-4544-9f55-8035164cefd5 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:26.118453 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 543/1082] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:48:25 2022] GET /identity/v3/users/b2f506f005d14e6aaee55b470fc661ee => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:26.125029 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ead81-6226-44c5-ae6e-4871d8aabd67 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:26.231268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ead81-6226-44c5-ae6e-4871d8aabd67 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:26.266752 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd488bea-4993-4c41-891c-a462d22246de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:26.267377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd488bea-4993-4c41-891c-a462d22246de None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:26.267655 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd488bea-4993-4c41-891c-a462d22246de None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:26.267927 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd488bea-4993-4c41-891c-a462d22246de None admin] PATH_INFO: `/v3/projects/8a5d3e15d53443178423add213e268d0` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:26.269059 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd488bea-4993-4c41-891c-a462d22246de None admin] RBAC: Authorizing `identity:get_project(project_id=8a5d3e15d53443178423add213e268d0)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:26.270489 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd488bea-4993-4c41-891c-a462d22246de None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:26.272297 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 540/1083] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:48:26 2022] GET /identity/v3/projects/8a5d3e15d53443178423add213e268d0 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:26.277961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a108e-c045-4544-9f55-8035164cefd5 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:26.377190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a108e-c045-4544-9f55-8035164cefd5 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:26.410507 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c29f01-f797-467b-8e14-f4360f3d0454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:26.411115 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c29f01-f797-467b-8e14-f4360f3d0454 None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:26.411903 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c29f01-f797-467b-8e14-f4360f3d0454 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:26.412211 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c29f01-f797-467b-8e14-f4360f3d0454 None admin] PATH_INFO: `/v3/projects/8a5d3e15d53443178423add213e268d0/users/b2f506f005d14e6aaee55b470fc661ee/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:26.414098 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c29f01-f797-467b-8e14-f4360f3d0454 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a5d3e15d53443178423add213e268d0, user_id=b2f506f005d14e6aaee55b470fc661ee, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:26.415526 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c29f01-f797-467b-8e14-f4360f3d0454 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:26.472416 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 544/1084] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Sep 9 20:48:26 2022] PUT /identity/v3/projects/8a5d3e15d53443178423add213e268d0/users/b2f506f005d14e6aaee55b470fc661ee/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:48:28.004454 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da34ca9-efe8-4eed-b47d-b742dfcdfc7b None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:28.004998 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da34ca9-efe8-4eed-b47d-b742dfcdfc7b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:28.005098 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da34ca9-efe8-4eed-b47d-b742dfcdfc7b None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:28.006540 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 541/1085] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:48:28.012214 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a607b6-a2f5-48a5-8e0b-16b0bdc413b2 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:28.012214 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a607b6-a2f5-48a5-8e0b-16b0bdc413b2 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:28.012675 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a607b6-a2f5-48a5-8e0b-16b0bdc413b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:28.030450 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-40a607b6-a2f5-48a5-8e0b-16b0bdc413b2 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:28.031614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-40a607b6-a2f5-48a5-8e0b-16b0bdc413b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:28.053940 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:28.097370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 545/1086] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:48:28.369447 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da34ca9-efe8-4eed-b47d-b742dfcdfc7b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:28.418159 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:28.472329 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da34ca9-efe8-4eed-b47d-b742dfcdfc7b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:28.509197 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c9b4b0-c359-4474-8fd7-41d399400299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:28.509824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9b4b0-c359-4474-8fd7-41d399400299 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:28.510097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9b4b0-c359-4474-8fd7-41d399400299 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:28.510363 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9b4b0-c359-4474-8fd7-41d399400299 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:28.520111 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c9b4b0-c359-4474-8fd7-41d399400299 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:48:28.525395 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-b5c9b4b0-c359-4474-8fd7-41d399400299 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 20:48:28.525395 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:28.525395 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:28.525395 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:28.525395 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:28.525395 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:28.525395 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:28.525395 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:28.525395 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:28.525395 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:28.525395 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:28.525395 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:28.525395 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:48:28.525395 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:48:28.525395 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:48:28.525395 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:48:28.526477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:48:28.526477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:48:28.526477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:48:28.526477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:48:28.526477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:28.526477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:28.526477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:28.526477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:28.526477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:28.526477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:28.526477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:28.526477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:28.526477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:28.526477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:28.526477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:28.526477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:28.527361 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:28.527361 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:28.527361 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:28.527361 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:28.527361 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:48:28.527361 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:48:28.527361 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:48:28.527361 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:48:28.527361 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:48:28.527361 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:48:28.527361 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 20:48:28.527361 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:28.527361 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 542/1087] 10.222.0.48 () {56 vars in 1120 bytes} [Fri Sep 9 20:48:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:28.532608 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a607b6-a2f5-48a5-8e0b-16b0bdc413b2 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:28.582240 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617b211-5dc1-490a-94aa-13a376f93fec None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:28.582520 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617b211-5dc1-490a-94aa-13a376f93fec None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:28.582794 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617b211-5dc1-490a-94aa-13a376f93fec None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:28.584728 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 543/1088] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:48:28.589528 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0a82f-3bdf-4d15-9f3b-7084c32b470b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:28.589808 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0a82f-3bdf-4d15-9f3b-7084c32b470b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:28.590077 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0a82f-3bdf-4d15-9f3b-7084c32b470b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:28.607908 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c0e0a82f-3bdf-4d15-9f3b-7084c32b470b None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:28.609346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c0e0a82f-3bdf-4d15-9f3b-7084c32b470b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:28.631414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a607b6-a2f5-48a5-8e0b-16b0bdc413b2 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:28.665824 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c56be-7f2a-4e3d-b7d0-9aeca4369491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:28.666672 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c56be-7f2a-4e3d-b7d0-9aeca4369491 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:28.666952 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c56be-7f2a-4e3d-b7d0-9aeca4369491 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:28.667224 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c56be-7f2a-4e3d-b7d0-9aeca4369491 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:28.668406 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c56be-7f2a-4e3d-b7d0-9aeca4369491 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:28.668941 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c56be-7f2a-4e3d-b7d0-9aeca4369491 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:28.671036 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c56be-7f2a-4e3d-b7d0-9aeca4369491 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:28.680341 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 546/1089] 10.222.0.48 () {56 vars in 1115 bytes} [Fri Sep 9 20:48:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:28.685532 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 544/1090] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:48:28.686285 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c56be-7f2a-4e3d-b7d0-9aeca4369491 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:28.709352 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:28.784054 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c56be-7f2a-4e3d-b7d0-9aeca4369491 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:28.822445 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1346dbe-f1d6-401b-959f-ff049a8eb33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:28.823081 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1346dbe-f1d6-401b-959f-ff049a8eb33e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:28.823356 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1346dbe-f1d6-401b-959f-ff049a8eb33e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:28.823628 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1346dbe-f1d6-401b-959f-ff049a8eb33e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:28.833874 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1346dbe-f1d6-401b-959f-ff049a8eb33e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:28.835296 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1346dbe-f1d6-401b-959f-ff049a8eb33e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:28.850404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-d1346dbe-f1d6-401b-959f-ff049a8eb33e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:48:28.850404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:28.850404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:28.850404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:28.850404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:28.850404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:28.850404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:28.850404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:28.850404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:28.850404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:28.850404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:28.850404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:28.850404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:48:28.850404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:48:28.850404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:48:28.850404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:48:28.851552 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:28.851552 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:28.851552 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:28.851552 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:28.851552 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:28.851552 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:28.851552 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:28.851552 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:28.851552 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:28.851552 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:28.851552 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:28.851552 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:28.851552 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:28.851552 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:28.851552 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:28.851552 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:28.851552 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:48:28.852503 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:48:28.852503 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:48:28.852503 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:28.852503 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 547/1091] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:48:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:48:28.857730 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e0a82f-3bdf-4d15-9f3b-7084c32b470b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:28.942118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:28.984866 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1346dbe-f1d6-401b-959f-ff049a8eb33e None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:29.004212 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e0a82f-3bdf-4d15-9f3b-7084c32b470b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:29.038942 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8935d-232c-4fbb-936f-43c740be9f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:29.039537 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8935d-232c-4fbb-936f-43c740be9f78 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:29.039809 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8935d-232c-4fbb-936f-43c740be9f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:29.040116 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8935d-232c-4fbb-936f-43c740be9f78 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:29.040759 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb8935d-232c-4fbb-936f-43c740be9f78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:29.041055 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb8935d-232c-4fbb-936f-43c740be9f78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:29.042419 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb8935d-232c-4fbb-936f-43c740be9f78 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:29.054453 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 545/1092] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:48:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:29.057618 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:29.059975 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8935d-232c-4fbb-936f-43c740be9f78 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:29.088454 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1346dbe-f1d6-401b-959f-ff049a8eb33e None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:29.122005 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e95e08-5fac-4636-a71e-59cf93e70078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:29.122630 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e95e08-5fac-4636-a71e-59cf93e70078 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:29.122906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e95e08-5fac-4636-a71e-59cf93e70078 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:29.123174 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e95e08-5fac-4636-a71e-59cf93e70078 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:29.133677 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e95e08-5fac-4636-a71e-59cf93e70078 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:48:29.142838 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-54e95e08-5fac-4636-a71e-59cf93e70078 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:48:29.142838 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:29.142838 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:29.142838 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:29.142838 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:29.142838 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:29.142838 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:29.142838 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:29.142838 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:29.142838 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:29.142838 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:29.142838 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:29.142838 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:48:29.142838 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:48:29.142838 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:48:29.142838 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:48:29.143949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:48:29.143949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:48:29.143949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:48:29.143949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:48:29.143949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:29.143949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:29.143949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:29.143949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:29.143949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:29.143949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:29.143949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:29.143949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:29.143949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:29.143949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:29.143949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:29.143949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:29.144851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:29.144851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:29.144851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:29.144851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:29.144851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:48:29.144851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:48:29.144851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:48:29.144851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:48:29.144851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:48:29.144851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:48:29.144851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:48:29.144851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:29.144851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 548/1093] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:48:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:48:29.150740 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e95e08-5fac-4636-a71e-59cf93e70078 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:29.157532 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:29.161010 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8935d-232c-4fbb-936f-43c740be9f78 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:29.197655 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5e29a0-5991-46f3-a802-de3354d43637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:29.198206 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e29a0-5991-46f3-a802-de3354d43637 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:29.198488 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e29a0-5991-46f3-a802-de3354d43637 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:29.198771 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e29a0-5991-46f3-a802-de3354d43637 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:29.205757 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5e29a0-5991-46f3-a802-de3354d43637 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:29.207207 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5e29a0-5991-46f3-a802-de3354d43637 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:29.218600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-6d5e29a0-5991-46f3-a802-de3354d43637 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:48:29.218600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:29.218600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:29.218600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:29.218600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:29.218600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:29.218600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:29.218600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:29.218600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:29.218600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:29.218600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:29.218600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:29.218600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:48:29.218600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:48:29.218600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:48:29.218600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:48:29.219620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:29.219620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:29.219620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:48:29.219620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:48:29.219620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:48:29.219620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:48:29.219620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:29.219620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:29.219620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:29.219620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:29.219620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:29.219620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:29.219620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:29.219620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:29.219620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:29.219620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:29.219620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:29.219620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:29.221171 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:29.221171 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:29.221171 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:48:29.221171 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:48:29.221171 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:48:29.221171 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:48:29.221171 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:48:29.221171 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:48:29.221171 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 20:48:29.221171 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:29.221171 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 546/1094] 10.222.0.48 () {56 vars in 1093 bytes} [Fri Sep 9 20:48:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:29.225908 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5e29a0-5991-46f3-a802-de3354d43637 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:29.257081 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e95e08-5fac-4636-a71e-59cf93e70078 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:29.285422 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:29.290834 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd4ef1-6c7c-4ab3-8e89-a40483865259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:29.291438 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd4ef1-6c7c-4ab3-8e89-a40483865259 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:29.291717 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd4ef1-6c7c-4ab3-8e89-a40483865259 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:29.291989 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd4ef1-6c7c-4ab3-8e89-a40483865259 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:29.292679 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd4ef1-6c7c-4ab3-8e89-a40483865259 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:29.293032 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd4ef1-6c7c-4ab3-8e89-a40483865259 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:29.294346 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd4ef1-6c7c-4ab3-8e89-a40483865259 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:29.303320 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 549/1095] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:48:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:29.309359 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd4ef1-6c7c-4ab3-8e89-a40483865259 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:29.324264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5e29a0-5991-46f3-a802-de3354d43637 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:29.357279 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a9602-2b39-4226-bdd4-0e93fb31d431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:29.357879 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a9602-2b39-4226-bdd4-0e93fb31d431 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:29.358165 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a9602-2b39-4226-bdd4-0e93fb31d431 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:29.358437 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a9602-2b39-4226-bdd4-0e93fb31d431 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:29.359161 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a9602-2b39-4226-bdd4-0e93fb31d431 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:29.359544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a9602-2b39-4226-bdd4-0e93fb31d431 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:29.360999 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a9602-2b39-4226-bdd4-0e93fb31d431 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:29.378411 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 547/1096] 10.222.0.48 () {56 vars in 1133 bytes} [Fri Sep 9 20:48:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:29.384127 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a9602-2b39-4226-bdd4-0e93fb31d431 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:29.400798 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:29.409365 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd4ef1-6c7c-4ab3-8e89-a40483865259 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:29.443114 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03614967-66a8-4fe5-bf11-c44597904868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:29.443735 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03614967-66a8-4fe5-bf11-c44597904868 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:29.450261 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03614967-66a8-4fe5-bf11-c44597904868 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:29.450261 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03614967-66a8-4fe5-bf11-c44597904868 None admin] PATH_INFO: `/v3/users/b2f506f005d14e6aaee55b470fc661ee` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:29.450261 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03614967-66a8-4fe5-bf11-c44597904868 None admin] RBAC: Authorizing `identity:get_user(user_id=b2f506f005d14e6aaee55b470fc661ee)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:29.450261 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03614967-66a8-4fe5-bf11-c44597904868 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:29.450261 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 550/1097] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:48:29 2022] GET /identity/v3/users/b2f506f005d14e6aaee55b470fc661ee => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:48:29.454191 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03614967-66a8-4fe5-bf11-c44597904868 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:29.484390 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a9602-2b39-4226-bdd4-0e93fb31d431 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:29.513987 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:29.517786 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74d68b-eb50-455c-b25f-bb903f6dda56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:29.518389 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74d68b-eb50-455c-b25f-bb903f6dda56 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:29.518663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74d68b-eb50-455c-b25f-bb903f6dda56 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:29.518931 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74d68b-eb50-455c-b25f-bb903f6dda56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:29.552279 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03614967-66a8-4fe5-bf11-c44597904868 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:29.584910 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74d68b-eb50-455c-b25f-bb903f6dda56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:29.585891 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e80d4-9aff-4d95-93e2-9884666a7872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:29.586356 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74d68b-eb50-455c-b25f-bb903f6dda56 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:29.586643 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e80d4-9aff-4d95-93e2-9884666a7872 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:29.586919 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e80d4-9aff-4d95-93e2-9884666a7872 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:29.587187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e80d4-9aff-4d95-93e2-9884666a7872 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:29.603190 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:29.652606 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9e80d4-9aff-4d95-93e2-9884666a7872 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:29.654040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9e80d4-9aff-4d95-93e2-9884666a7872 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:29.682974 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 548/1098] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:48:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 20:48:29.689045 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74d68b-eb50-455c-b25f-bb903f6dda56 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:29.712901 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:29.755706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 551/1099] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:48:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 20:48:29.756745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e80d4-9aff-4d95-93e2-9884666a7872 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:29.783900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74d68b-eb50-455c-b25f-bb903f6dda56 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:29.801426 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:29.819114 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a88465-9423-4493-86f3-e9be55d996d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:29.819740 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a88465-9423-4493-86f3-e9be55d996d5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:29.820022 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a88465-9423-4493-86f3-e9be55d996d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:29.820323 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a88465-9423-4493-86f3-e9be55d996d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:29.829961 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a88465-9423-4493-86f3-e9be55d996d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:29.831389 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a88465-9423-4493-86f3-e9be55d996d5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:29.846525 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-f4a88465-9423-4493-86f3-e9be55d996d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:48:29.846525 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:29.846525 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:29.846525 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:29.846525 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:29.846525 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:29.846525 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:29.846525 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:29.846525 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:29.846525 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:29.846525 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:29.846525 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:29.846525 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:48:29.846525 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:48:29.846525 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:48:29.846525 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:48:29.847483 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:29.847483 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:29.847483 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:29.847483 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:29.847483 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:29.847483 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:29.847483 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:29.847483 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:29.847483 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:29.847483 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:29.847483 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:29.847483 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:29.847483 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:29.847483 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:29.847483 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:29.847483 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:29.847483 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:48:29.848431 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:48:29.848431 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:48:29.848431 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:29.848431 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 549/1100] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:48:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:29.852907 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e80d4-9aff-4d95-93e2-9884666a7872 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:29.853708 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a88465-9423-4493-86f3-e9be55d996d5 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:29.886630 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a09faf-3ebd-4edf-af3d-eff5d4325f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:29.887955 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a09faf-3ebd-4edf-af3d-eff5d4325f30 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:29.888265 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a09faf-3ebd-4edf-af3d-eff5d4325f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:29.888531 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a09faf-3ebd-4edf-af3d-eff5d4325f30 None admin] PATH_INFO: `/v3/projects/8a5d3e15d53443178423add213e268d0` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:29.889634 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a09faf-3ebd-4edf-af3d-eff5d4325f30 None admin] RBAC: Authorizing `identity:get_project(project_id=8a5d3e15d53443178423add213e268d0)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:29.891043 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a09faf-3ebd-4edf-af3d-eff5d4325f30 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:29.892668 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 552/1101] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:48:29 2022] GET /identity/v3/projects/8a5d3e15d53443178423add213e268d0 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:29.899151 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a09faf-3ebd-4edf-af3d-eff5d4325f30 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:29.927245 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:29.951782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a88465-9423-4493-86f3-e9be55d996d5 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:29.985027 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee860b-252c-4980-ab05-1ba990bcb5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:29.985550 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee860b-252c-4980-ab05-1ba990bcb5f9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:29.985840 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee860b-252c-4980-ab05-1ba990bcb5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:29.986115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee860b-252c-4980-ab05-1ba990bcb5f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:29.986759 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ee860b-252c-4980-ab05-1ba990bcb5f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:29.987064 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ee860b-252c-4980-ab05-1ba990bcb5f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:29.988472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ee860b-252c-4980-ab05-1ba990bcb5f9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:30.000094 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 550/1102] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:48:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:48:30.001064 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a09faf-3ebd-4edf-af3d-eff5d4325f30 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:30.005964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee860b-252c-4980-ab05-1ba990bcb5f9 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:30.033388 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:30.037736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe4aa1-fafb-41ae-8432-fb3bee9a7a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:30.038343 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe4aa1-fafb-41ae-8432-fb3bee9a7a12 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:30.038625 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe4aa1-fafb-41ae-8432-fb3bee9a7a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:30.038898 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe4aa1-fafb-41ae-8432-fb3bee9a7a12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:30.039620 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe4aa1-fafb-41ae-8432-fb3bee9a7a12 None admin] RBAC: Adding query filter params (role.id=fcbf519fbc84458f913671a5a1d0914c, scope.project.id=8a5d3e15d53443178423add213e268d0, user.id=b2f506f005d14e6aaee55b470fc661ee) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:30.039934 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe4aa1-fafb-41ae-8432-fb3bee9a7a12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:30.041406 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe4aa1-fafb-41ae-8432-fb3bee9a7a12 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:30.047069 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 553/1103] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:48:29 2022] GET /identity/v3/role_assignments?user.id=b2f506f005d14e6aaee55b470fc661ee&scope.project.id=8a5d3e15d53443178423add213e268d0&role.id=fcbf519fbc84458f913671a5a1d0914c => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:48:30.107669 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee860b-252c-4980-ab05-1ba990bcb5f9 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:30.144779 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637a834-6a3f-4804-8090-b9eda9c4d977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:30.145430 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637a834-6a3f-4804-8090-b9eda9c4d977 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:30.146254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637a834-6a3f-4804-8090-b9eda9c4d977 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:30.146534 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637a834-6a3f-4804-8090-b9eda9c4d977 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:30.153697 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637a834-6a3f-4804-8090-b9eda9c4d977 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:30.155160 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637a834-6a3f-4804-8090-b9eda9c4d977 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:30.167104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-e637a834-6a3f-4804-8090-b9eda9c4d977 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:48:30.167104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:30.167104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:30.167104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:30.167104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:30.167104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:30.167104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:30.167104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:30.167104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:30.167104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:30.167104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:30.167104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:30.167104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:48:30.167104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:48:30.167104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:48:30.168691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:48:30.168691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:30.168691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:30.168691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:30.168691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:30.168691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:30.168691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:30.168691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:30.168691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:30.168691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:30.168691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:30.168691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:30.168691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:30.168691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:30.168691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:30.168691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:30.168691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:30.169721 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:48:30.169721 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:48:30.169721 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:48:30.169721 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:48:30.169721 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:48:30.169721 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:48:30.169721 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:48:30.169721 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:30.169721 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 551/1104] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:48:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:48:30.179616 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe4aa1-fafb-41ae-8432-fb3bee9a7a12 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:30.275215 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:30.282677 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe4aa1-fafb-41ae-8432-fb3bee9a7a12 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:30.316029 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c119b5-515d-4cbf-a3b7-1875ad46d978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:30.316662 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c119b5-515d-4cbf-a3b7-1875ad46d978 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:30.316942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c119b5-515d-4cbf-a3b7-1875ad46d978 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:30.317215 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c119b5-515d-4cbf-a3b7-1875ad46d978 None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:30.318575 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c119b5-515d-4cbf-a3b7-1875ad46d978 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:30.318878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c119b5-515d-4cbf-a3b7-1875ad46d978 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:30.320318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c119b5-515d-4cbf-a3b7-1875ad46d978 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:30.331473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 554/1105] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:48:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:30.337472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637a834-6a3f-4804-8090-b9eda9c4d977 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:30.421959 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:30.434405 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637a834-6a3f-4804-8090-b9eda9c4d977 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:30.468840 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d01e05b-d4d1-478d-812a-696010b82bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:30.469439 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01e05b-d4d1-478d-812a-696010b82bad None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:30.469717 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01e05b-d4d1-478d-812a-696010b82bad None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:30.469985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01e05b-d4d1-478d-812a-696010b82bad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:30.470702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d01e05b-d4d1-478d-812a-696010b82bad None admin] RBAC: Adding query filter params (role.id=63740f83debd49b6a81f640fe2a07ce2, scope.project.id=91319aa782334fb9930770029b74e84a, user.id=613b845a9b4d432daf8d478fc3970708) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:30.471011 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d01e05b-d4d1-478d-812a-696010b82bad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:30.472464 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d01e05b-d4d1-478d-812a-696010b82bad None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:30.478144 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 552/1106] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:48:30 2022] GET /identity/v3/role_assignments?user.id=613b845a9b4d432daf8d478fc3970708&scope.project.id=91319aa782334fb9930770029b74e84a&role.id=63740f83debd49b6a81f640fe2a07ce2 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:32.238087 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae27871-478f-4668-8489-4162986dbe64 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:32.238670 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae27871-478f-4668-8489-4162986dbe64 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:32.238670 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae27871-478f-4668-8489-4162986dbe64 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:32.240626 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 555/1107] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:48:32.245841 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c2f42-f62b-4d1d-8fed-7f6a8fa44f82 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:32.246299 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c2f42-f62b-4d1d-8fed-7f6a8fa44f82 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:32.246402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c2f42-f62b-4d1d-8fed-7f6a8fa44f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:32.264272 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-041c2f42-f62b-4d1d-8fed-7f6a8fa44f82 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:32.265396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-041c2f42-f62b-4d1d-8fed-7f6a8fa44f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:32.331016 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 553/1108] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:48:32.598359 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae27871-478f-4668-8489-4162986dbe64 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:32.695882 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae27871-478f-4668-8489-4162986dbe64 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:32.729168 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b0284f-cc32-47e1-99dd-1c1990b45025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:32.729779 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0284f-cc32-47e1-99dd-1c1990b45025 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:32.730059 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0284f-cc32-47e1-99dd-1c1990b45025 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:32.730332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0284f-cc32-47e1-99dd-1c1990b45025 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:32.731430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b0284f-cc32-47e1-99dd-1c1990b45025 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:32.732898 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b0284f-cc32-47e1-99dd-1c1990b45025 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:32.734421 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 556/1109] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:48:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:32.740001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c2f42-f62b-4d1d-8fed-7f6a8fa44f82 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:32.816652 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:32.834013 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c2f42-f62b-4d1d-8fed-7f6a8fa44f82 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:32.866710 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1627b-7f8a-420b-a31c-cb056a0c7b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:32.867234 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1627b-7f8a-420b-a31c-cb056a0c7b6c None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:32.867507 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1627b-7f8a-420b-a31c-cb056a0c7b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:32.867772 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1627b-7f8a-420b-a31c-cb056a0c7b6c None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:32.868445 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f1627b-7f8a-420b-a31c-cb056a0c7b6c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:32.869856 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f1627b-7f8a-420b-a31c-cb056a0c7b6c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:32.980475 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 554/1110] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Sep 9 20:48:32 2022] POST /identity/v3/users => generated 286 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:48:35.142393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da073740-6ce1-499d-b5f3-695d2b81b2b6 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:35.142930 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da073740-6ce1-499d-b5f3-695d2b81b2b6 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:35.142930 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da073740-6ce1-499d-b5f3-695d2b81b2b6 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:35.144349 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 557/1111] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:48:35.149638 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e18e76-c39f-4988-b289-bcd1c325e69c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:35.150065 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e18e76-c39f-4988-b289-bcd1c325e69c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:35.150167 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e18e76-c39f-4988-b289-bcd1c325e69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:35.168289 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-25e18e76-c39f-4988-b289-bcd1c325e69c None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:35.169447 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-25e18e76-c39f-4988-b289-bcd1c325e69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:35.232545 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 555/1112] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:48:35.501903 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da073740-6ce1-499d-b5f3-695d2b81b2b6 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:35.526200 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:35.601228 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da073740-6ce1-499d-b5f3-695d2b81b2b6 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:35.633567 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e7988-f98b-47ba-8497-c270ca274a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:35.634176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e7988-f98b-47ba-8497-c270ca274a11 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:35.634453 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e7988-f98b-47ba-8497-c270ca274a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:35.634722 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e7988-f98b-47ba-8497-c270ca274a11 None admin] PATH_INFO: `/v3/roles/13072ad4b81649459f59b8137db492ec` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:35.636119 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2e7988-f98b-47ba-8497-c270ca274a11 None admin] RBAC: Authorizing `identity:get_role(role_id=13072ad4b81649459f59b8137db492ec)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:35.637522 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2e7988-f98b-47ba-8497-c270ca274a11 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:35.638815 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 558/1113] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:48:35 2022] GET /identity/v3/roles/13072ad4b81649459f59b8137db492ec => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:35.644509 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e18e76-c39f-4988-b289-bcd1c325e69c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:35.707554 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:35.739368 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e18e76-c39f-4988-b289-bcd1c325e69c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:35.771866 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55ec19-957f-4325-b58c-d49c6fb7ece0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:35.772447 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55ec19-957f-4325-b58c-d49c6fb7ece0 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:35.772722 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55ec19-957f-4325-b58c-d49c6fb7ece0 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:35.772989 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55ec19-957f-4325-b58c-d49c6fb7ece0 None admin] PATH_INFO: `/v3/users/b67ea0272f2d4b0b8e35f37e5cba71a1` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:35.786017 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55ec19-957f-4325-b58c-d49c6fb7ece0 None admin] RBAC: Authorizing `identity:get_user(user_id=b67ea0272f2d4b0b8e35f37e5cba71a1)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:35.791351 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55ec19-957f-4325-b58c-d49c6fb7ece0 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:35.791351 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 556/1114] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:48:35 2022] GET /identity/v3/users/b67ea0272f2d4b0b8e35f37e5cba71a1 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:35.795266 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e7988-f98b-47ba-8497-c270ca274a11 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:35.883032 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:35.892534 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e7988-f98b-47ba-8497-c270ca274a11 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:35.925562 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a3d20-cd35-41d2-8cad-3f955f322b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:35.926159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a3d20-cd35-41d2-8cad-3f955f322b31 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:35.926439 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a3d20-cd35-41d2-8cad-3f955f322b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:35.926710 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a3d20-cd35-41d2-8cad-3f955f322b31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:35.992177 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a3d20-cd35-41d2-8cad-3f955f322b31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:35.993627 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a3d20-cd35-41d2-8cad-3f955f322b31 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:36.079770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:36.090102 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 559/1115] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:48:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:48:36.096183 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55ec19-957f-4325-b58c-d49c6fb7ece0 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:36.190942 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55ec19-957f-4325-b58c-d49c6fb7ece0 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:36.223813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:36.232437 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f3fa27-9c49-455d-b675-5194ca8f69cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:36.233072 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3fa27-9c49-455d-b675-5194ca8f69cb None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:36.233356 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3fa27-9c49-455d-b675-5194ca8f69cb None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:36.233620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3fa27-9c49-455d-b675-5194ca8f69cb None admin] PATH_INFO: `/v3/projects/8a5d3e15d53443178423add213e268d0` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:36.234725 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f3fa27-9c49-455d-b675-5194ca8f69cb None admin] RBAC: Authorizing `identity:get_project(project_id=8a5d3e15d53443178423add213e268d0)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:36.236189 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f3fa27-9c49-455d-b675-5194ca8f69cb None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:36.237793 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 557/1116] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:48:36 2022] GET /identity/v3/projects/8a5d3e15d53443178423add213e268d0 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:36.244501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a3d20-cd35-41d2-8cad-3f955f322b31 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:36.341253 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a3d20-cd35-41d2-8cad-3f955f322b31 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:36.373352 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae1db9-c298-45b8-b2fc-045e9df73882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:36.373935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae1db9-c298-45b8-b2fc-045e9df73882 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:36.374210 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae1db9-c298-45b8-b2fc-045e9df73882 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:36.374478 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae1db9-c298-45b8-b2fc-045e9df73882 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:36.375191 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ae1db9-c298-45b8-b2fc-045e9df73882 None admin] RBAC: Adding query filter params (role.id=13072ad4b81649459f59b8137db492ec, scope.project.id=8a5d3e15d53443178423add213e268d0, user.id=b67ea0272f2d4b0b8e35f37e5cba71a1) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:36.375496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ae1db9-c298-45b8-b2fc-045e9df73882 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:36.376937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ae1db9-c298-45b8-b2fc-045e9df73882 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:36.382604 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 560/1117] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:48:36 2022] GET /identity/v3/role_assignments?user.id=b67ea0272f2d4b0b8e35f37e5cba71a1&scope.project.id=8a5d3e15d53443178423add213e268d0&role.id=13072ad4b81649459f59b8137db492ec => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:38.459502 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664605b-e404-4a68-88bb-de08ff6c62f3 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:38.460052 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664605b-e404-4a68-88bb-de08ff6c62f3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:38.460052 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664605b-e404-4a68-88bb-de08ff6c62f3 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:38.461382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 558/1118] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:48:38.466517 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec464d7-93c4-4bc5-8dd1-f99439e0b206 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:38.466931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec464d7-93c4-4bc5-8dd1-f99439e0b206 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:38.467056 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec464d7-93c4-4bc5-8dd1-f99439e0b206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:38.485177 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-0ec464d7-93c4-4bc5-8dd1-f99439e0b206 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:38.486382 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0ec464d7-93c4-4bc5-8dd1-f99439e0b206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:38.533541 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:38.550387 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 561/1119] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:48:38.821585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1664605b-e404-4a68-88bb-de08ff6c62f3 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:38.919713 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1664605b-e404-4a68-88bb-de08ff6c62f3 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:38.952342 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7138fda-26bb-420c-abda-a3da6fdd558c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:38.952965 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7138fda-26bb-420c-abda-a3da6fdd558c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:38.953242 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7138fda-26bb-420c-abda-a3da6fdd558c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:38.953513 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7138fda-26bb-420c-abda-a3da6fdd558c None admin] PATH_INFO: `/v3/roles/13072ad4b81649459f59b8137db492ec` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:38.954883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7138fda-26bb-420c-abda-a3da6fdd558c None admin] RBAC: Authorizing `identity:get_role(role_id=13072ad4b81649459f59b8137db492ec)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:38.956283 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7138fda-26bb-420c-abda-a3da6fdd558c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:38.957583 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 559/1120] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:48:38 2022] GET /identity/v3/roles/13072ad4b81649459f59b8137db492ec => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:38.964171 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec464d7-93c4-4bc5-8dd1-f99439e0b206 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:39.058531 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec464d7-93c4-4bc5-8dd1-f99439e0b206 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:39.092172 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cfaa45-453e-4b82-9042-fe069c16da45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:39.092728 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfaa45-453e-4b82-9042-fe069c16da45 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:39.093000 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfaa45-453e-4b82-9042-fe069c16da45 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:39.093266 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfaa45-453e-4b82-9042-fe069c16da45 None admin] PATH_INFO: `/v3/users/b67ea0272f2d4b0b8e35f37e5cba71a1` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:39.094352 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cfaa45-453e-4b82-9042-fe069c16da45 None admin] RBAC: Authorizing `identity:get_user(user_id=b67ea0272f2d4b0b8e35f37e5cba71a1)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:39.095754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cfaa45-453e-4b82-9042-fe069c16da45 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:39.097310 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 562/1121] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:48:38 2022] GET /identity/v3/users/b67ea0272f2d4b0b8e35f37e5cba71a1 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:39.103197 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7138fda-26bb-420c-abda-a3da6fdd558c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:39.162693 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:39.199934 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7138fda-26bb-420c-abda-a3da6fdd558c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:39.241374 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606f9cf-2fe2-4202-a2a0-4b0a2f91de0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:39.242004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606f9cf-2fe2-4202-a2a0-4b0a2f91de0d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:39.242284 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606f9cf-2fe2-4202-a2a0-4b0a2f91de0d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:39.242557 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606f9cf-2fe2-4202-a2a0-4b0a2f91de0d None admin] PATH_INFO: `/v3/projects/8a5d3e15d53443178423add213e268d0` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:39.243696 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e606f9cf-2fe2-4202-a2a0-4b0a2f91de0d None admin] RBAC: Authorizing `identity:get_project(project_id=8a5d3e15d53443178423add213e268d0)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:39.245162 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e606f9cf-2fe2-4202-a2a0-4b0a2f91de0d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:39.246780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 560/1122] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:48:39 2022] GET /identity/v3/projects/8a5d3e15d53443178423add213e268d0 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:39.252831 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cfaa45-453e-4b82-9042-fe069c16da45 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:39.303369 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:39.348469 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cfaa45-453e-4b82-9042-fe069c16da45 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:39.381104 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58adfd-746b-49be-bacc-2f8d155545de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:39.381733 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58adfd-746b-49be-bacc-2f8d155545de None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:39.382007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58adfd-746b-49be-bacc-2f8d155545de None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:39.382272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58adfd-746b-49be-bacc-2f8d155545de None admin] PATH_INFO: `/v3/projects/8a5d3e15d53443178423add213e268d0/users/b67ea0272f2d4b0b8e35f37e5cba71a1/roles/13072ad4b81649459f59b8137db492ec` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:39.384367 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58adfd-746b-49be-bacc-2f8d155545de None admin] RBAC: Authorizing `identity:create_grant(project_id=8a5d3e15d53443178423add213e268d0, user_id=b67ea0272f2d4b0b8e35f37e5cba71a1, role_id=13072ad4b81649459f59b8137db492ec)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:39.385776 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58adfd-746b-49be-bacc-2f8d155545de None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:39.432949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 563/1123] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Sep 9 20:48:39 2022] PUT /identity/v3/projects/8a5d3e15d53443178423add213e268d0/users/b67ea0272f2d4b0b8e35f37e5cba71a1/roles/13072ad4b81649459f59b8137db492ec => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:48:41.531301 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe401292-74d2-469c-bc48-2b999fa42bf7 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:41.531864 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe401292-74d2-469c-bc48-2b999fa42bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:41.531864 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe401292-74d2-469c-bc48-2b999fa42bf7 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:41.533223 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 561/1124] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:48:41.538483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa0e78-1538-49bd-978c-48cc1c340c5c None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:41.538908 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa0e78-1538-49bd-978c-48cc1c340c5c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:41.539033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa0e78-1538-49bd-978c-48cc1c340c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:41.557311 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-40aa0e78-1538-49bd-978c-48cc1c340c5c None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:41.558526 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-40aa0e78-1538-49bd-978c-48cc1c340c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:41.622606 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 564/1125] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:48:41.889523 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe401292-74d2-469c-bc48-2b999fa42bf7 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:41.988156 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe401292-74d2-469c-bc48-2b999fa42bf7 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:42.016679 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:42.022490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1bb90-fc95-42b3-8fe3-b291aa4ec2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:42.023124 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1bb90-fc95-42b3-8fe3-b291aa4ec2d2 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:42.023406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1bb90-fc95-42b3-8fe3-b291aa4ec2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:42.023681 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1bb90-fc95-42b3-8fe3-b291aa4ec2d2 None admin] PATH_INFO: `/v3/roles/13072ad4b81649459f59b8137db492ec` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:42.025117 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa1bb90-fc95-42b3-8fe3-b291aa4ec2d2 None admin] RBAC: Authorizing `identity:get_role(role_id=13072ad4b81649459f59b8137db492ec)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:42.026570 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa1bb90-fc95-42b3-8fe3-b291aa4ec2d2 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:42.027880 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 562/1126] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:48:41 2022] GET /identity/v3/roles/13072ad4b81649459f59b8137db492ec => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:42.033637 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aa0e78-1538-49bd-978c-48cc1c340c5c None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:42.075563 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:42.130928 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aa0e78-1538-49bd-978c-48cc1c340c5c None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:42.164337 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45fa19-2514-4113-b27f-ce1b562c4a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:42.164957 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45fa19-2514-4113-b27f-ce1b562c4a3e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:42.165235 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45fa19-2514-4113-b27f-ce1b562c4a3e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:42.165506 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45fa19-2514-4113-b27f-ce1b562c4a3e None admin] PATH_INFO: `/v3/users/b67ea0272f2d4b0b8e35f37e5cba71a1` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:42.166609 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45fa19-2514-4113-b27f-ce1b562c4a3e None admin] RBAC: Authorizing `identity:get_user(user_id=b67ea0272f2d4b0b8e35f37e5cba71a1)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:42.168047 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45fa19-2514-4113-b27f-ce1b562c4a3e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:42.169642 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 565/1127] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:48:42 2022] GET /identity/v3/users/b67ea0272f2d4b0b8e35f37e5cba71a1 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:42.175386 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1bb90-fc95-42b3-8fe3-b291aa4ec2d2 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:42.275952 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1bb90-fc95-42b3-8fe3-b291aa4ec2d2 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:42.308775 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f43510-d986-46ce-a886-d867816c2116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:42.309372 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f43510-d986-46ce-a886-d867816c2116 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:42.309649 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f43510-d986-46ce-a886-d867816c2116 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:42.309921 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f43510-d986-46ce-a886-d867816c2116 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:42.371685 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:42.374978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f43510-d986-46ce-a886-d867816c2116 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:42.376480 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f43510-d986-46ce-a886-d867816c2116 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:42.471650 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 563/1128] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:48:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:48:42.478108 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45fa19-2514-4113-b27f-ce1b562c4a3e None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:42.571944 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45fa19-2514-4113-b27f-ce1b562c4a3e None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:42.582228 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:42.605638 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e4e9f-c067-4b05-982b-8ecdbcb63dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:42.606259 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e4e9f-c067-4b05-982b-8ecdbcb63dd3 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:42.606536 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e4e9f-c067-4b05-982b-8ecdbcb63dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:42.606805 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e4e9f-c067-4b05-982b-8ecdbcb63dd3 None admin] PATH_INFO: `/v3/projects/8a5d3e15d53443178423add213e268d0` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:42.607903 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e4e9f-c067-4b05-982b-8ecdbcb63dd3 None admin] RBAC: Authorizing `identity:get_project(project_id=8a5d3e15d53443178423add213e268d0)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:42.609354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e4e9f-c067-4b05-982b-8ecdbcb63dd3 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:42.610945 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 566/1129] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:48:42 2022] GET /identity/v3/projects/8a5d3e15d53443178423add213e268d0 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:42.617043 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f43510-d986-46ce-a886-d867816c2116 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:42.710452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:42.714232 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f43510-d986-46ce-a886-d867816c2116 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:42.746975 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1578714-602a-4288-9afc-cb102155ba86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:42.747509 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1578714-602a-4288-9afc-cb102155ba86 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:42.747788 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1578714-602a-4288-9afc-cb102155ba86 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:42.748059 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1578714-602a-4288-9afc-cb102155ba86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:42.748807 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1578714-602a-4288-9afc-cb102155ba86 None admin] RBAC: Adding query filter params (role.id=13072ad4b81649459f59b8137db492ec, scope.project.id=8a5d3e15d53443178423add213e268d0, user.id=b67ea0272f2d4b0b8e35f37e5cba71a1) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:42.749119 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1578714-602a-4288-9afc-cb102155ba86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:42.750549 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1578714-602a-4288-9afc-cb102155ba86 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:42.756652 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 564/1130] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:48:42 2022] GET /identity/v3/role_assignments?user.id=b67ea0272f2d4b0b8e35f37e5cba71a1&scope.project.id=8a5d3e15d53443178423add213e268d0&role.id=13072ad4b81649459f59b8137db492ec => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:44.881942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747eec62-1d0f-46c6-93b4-c09522af6040 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:44.882474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747eec62-1d0f-46c6-93b4-c09522af6040 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:44.882474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747eec62-1d0f-46c6-93b4-c09522af6040 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:44.884085 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 567/1131] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:48:44.889274 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935233c-522c-41e5-a7d2-a123a005907c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:44.889714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935233c-522c-41e5-a7d2-a123a005907c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:44.889818 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935233c-522c-41e5-a7d2-a123a005907c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:44.908439 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-0935233c-522c-41e5-a7d2-a123a005907c None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:44.909619 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0935233c-522c-41e5-a7d2-a123a005907c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:44.976155 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 565/1132] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:48:45.251575 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747eec62-1d0f-46c6-93b4-c09522af6040 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:45.348693 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747eec62-1d0f-46c6-93b4-c09522af6040 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:45.381522 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf0d22-4c76-4121-87bd-5fc6435b3173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:45.382123 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf0d22-4c76-4121-87bd-5fc6435b3173 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:45.382507 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf0d22-4c76-4121-87bd-5fc6435b3173 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:45.382916 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf0d22-4c76-4121-87bd-5fc6435b3173 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:45.384540 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cf0d22-4c76-4121-87bd-5fc6435b3173 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:45.386853 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cf0d22-4c76-4121-87bd-5fc6435b3173 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:45.388833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 568/1133] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:48:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:45.394695 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935233c-522c-41e5-a7d2-a123a005907c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:45.482048 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:45.490905 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935233c-522c-41e5-a7d2-a123a005907c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:45.523466 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957c7b4-9a8c-4fc5-845b-46065a52daec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:45.523999 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957c7b4-9a8c-4fc5-845b-46065a52daec None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:45.524307 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957c7b4-9a8c-4fc5-845b-46065a52daec None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:45.524581 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957c7b4-9a8c-4fc5-845b-46065a52daec None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:45.525222 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957c7b4-9a8c-4fc5-845b-46065a52daec None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:45.526660 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957c7b4-9a8c-4fc5-845b-46065a52daec None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:45.634037 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 566/1134] 10.222.0.48 () {60 vars in 1141 bytes} [Fri Sep 9 20:48:45 2022] POST /identity/v3/projects => generated 311 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:48:47.781881 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb9f3e-e87b-4595-baea-efcb11359bfd None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:47.782449 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb9f3e-e87b-4595-baea-efcb11359bfd None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:47.782449 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb9f3e-e87b-4595-baea-efcb11359bfd None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:47.783730 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 569/1135] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:48:47.789499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87509e84-663e-4d42-b031-9eb7bb691c6d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:47.789898 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87509e84-663e-4d42-b031-9eb7bb691c6d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:47.790044 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87509e84-663e-4d42-b031-9eb7bb691c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:47.807651 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-87509e84-663e-4d42-b031-9eb7bb691c6d None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:47.808850 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-87509e84-663e-4d42-b031-9eb7bb691c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:47.872612 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 567/1136] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:48:48.142197 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb9f3e-e87b-4595-baea-efcb11359bfd None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:48.188948 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:48.242208 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb9f3e-e87b-4595-baea-efcb11359bfd None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:48.280282 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b56e08-a520-4d21-8e6d-c7da0ffe504e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:48.280911 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b56e08-a520-4d21-8e6d-c7da0ffe504e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:48.281191 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b56e08-a520-4d21-8e6d-c7da0ffe504e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:48.281463 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b56e08-a520-4d21-8e6d-c7da0ffe504e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:48.282567 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b56e08-a520-4d21-8e6d-c7da0ffe504e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:48.283982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b56e08-a520-4d21-8e6d-c7da0ffe504e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:48.285558 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 570/1137] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:48:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:48.291239 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87509e84-663e-4d42-b031-9eb7bb691c6d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:48.363238 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:48.388962 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87509e84-663e-4d42-b031-9eb7bb691c6d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:48.421492 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49269d8-20a0-41d2-a653-6367e5a1d1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:48.422083 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49269d8-20a0-41d2-a653-6367e5a1d1d3 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:48.422358 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49269d8-20a0-41d2-a653-6367e5a1d1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:48.422625 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49269d8-20a0-41d2-a653-6367e5a1d1d3 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:48.423270 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49269d8-20a0-41d2-a653-6367e5a1d1d3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:48.424752 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49269d8-20a0-41d2-a653-6367e5a1d1d3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:48.495689 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 568/1138] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Sep 9 20:48:48 2022] POST /identity/v3/users => generated 286 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:48:50.664819 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425dfa85-6723-4b01-ade7-50b698a82a23 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:50.665383 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425dfa85-6723-4b01-ade7-50b698a82a23 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:50.665383 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425dfa85-6723-4b01-ade7-50b698a82a23 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:50.666654 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 571/1139] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:48:50.671781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850012a-6e2d-4d6e-97d7-9f9b6b88f3dc None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:50.672198 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850012a-6e2d-4d6e-97d7-9f9b6b88f3dc None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:50.672363 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850012a-6e2d-4d6e-97d7-9f9b6b88f3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:50.690953 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5850012a-6e2d-4d6e-97d7-9f9b6b88f3dc None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:50.692193 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5850012a-6e2d-4d6e-97d7-9f9b6b88f3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:50.755747 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 569/1140] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:48:51.025159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425dfa85-6723-4b01-ade7-50b698a82a23 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:51.124192 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425dfa85-6723-4b01-ade7-50b698a82a23 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:51.138229 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:51.157903 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ea386-5084-412b-a38a-28431f45a65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:51.158528 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ea386-5084-412b-a38a-28431f45a65b None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:51.158804 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ea386-5084-412b-a38a-28431f45a65b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:51.159072 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ea386-5084-412b-a38a-28431f45a65b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:51.168818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021ea386-5084-412b-a38a-28431f45a65b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:48:51.173612 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-021ea386-5084-412b-a38a-28431f45a65b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:48:51.173612 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:51.173612 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:51.173612 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:51.173612 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:51.173612 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:51.173612 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:51.173612 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:51.173612 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:51.173612 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:51.173612 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:51.173612 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:51.173612 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:48:51.173612 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:48:51.173612 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:48:51.173612 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:48:51.174745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:48:51.174745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:48:51.174745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:48:51.174745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:48:51.174745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:51.174745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:51.174745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:51.174745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:51.174745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:51.174745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:51.174745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:51.174745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:51.174745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:51.174745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:51.174745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:51.174745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:51.175776 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:51.175776 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:51.175776 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:51.175776 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:51.175776 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:48:51.175776 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:48:51.175776 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:48:51.175776 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:48:51.175776 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:48:51.175776 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:48:51.175776 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:48:51.175776 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:51.175776 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 572/1141] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:48:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:51.182145 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850012a-6e2d-4d6e-97d7-9f9b6b88f3dc None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:51.238452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:51.277831 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850012a-6e2d-4d6e-97d7-9f9b6b88f3dc None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:51.310202 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743357f-f209-4403-b2b6-4d71cb1e034a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:51.310793 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743357f-f209-4403-b2b6-4d71cb1e034a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:51.311075 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743357f-f209-4403-b2b6-4d71cb1e034a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:51.311344 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743357f-f209-4403-b2b6-4d71cb1e034a None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:51.311998 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743357f-f209-4403-b2b6-4d71cb1e034a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:51.312579 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743357f-f209-4403-b2b6-4d71cb1e034a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:51.313934 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743357f-f209-4403-b2b6-4d71cb1e034a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:51.322150 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 570/1142] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:48:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:51.327851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ea386-5084-412b-a38a-28431f45a65b None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:51.424007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ea386-5084-412b-a38a-28431f45a65b None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:51.456998 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04490ecd-7f2a-47db-a2cf-b0221fd519b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:51.457666 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04490ecd-7f2a-47db-a2cf-b0221fd519b4 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:51.457940 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04490ecd-7f2a-47db-a2cf-b0221fd519b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:51.458206 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04490ecd-7f2a-47db-a2cf-b0221fd519b4 None admin] PATH_INFO: `/v3/users/57c8811a06dd474ea802e187d1ac0299` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:51.470569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04490ecd-7f2a-47db-a2cf-b0221fd519b4 None admin] RBAC: Authorizing `identity:get_user(user_id=57c8811a06dd474ea802e187d1ac0299)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:51.471990 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04490ecd-7f2a-47db-a2cf-b0221fd519b4 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:51.473617 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 573/1143] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:48:51 2022] GET /identity/v3/users/57c8811a06dd474ea802e187d1ac0299 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:51.479392 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743357f-f209-4403-b2b6-4d71cb1e034a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:51.576678 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743357f-f209-4403-b2b6-4d71cb1e034a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:51.597757 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:51.608841 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90658dfd-4ec8-48c0-b3a5-fa3aaf3a89df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:51.609437 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90658dfd-4ec8-48c0-b3a5-fa3aaf3a89df None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:51.612513 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90658dfd-4ec8-48c0-b3a5-fa3aaf3a89df None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:51.612513 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90658dfd-4ec8-48c0-b3a5-fa3aaf3a89df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:51.674786 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90658dfd-4ec8-48c0-b3a5-fa3aaf3a89df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:51.676849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90658dfd-4ec8-48c0-b3a5-fa3aaf3a89df None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:51.770287 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 571/1144] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:48:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:48:51.776218 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04490ecd-7f2a-47db-a2cf-b0221fd519b4 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:51.806581 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:51.871535 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04490ecd-7f2a-47db-a2cf-b0221fd519b4 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:51.902989 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f1e914-3ebb-4b84-b359-f02af0b244d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:51.903595 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1e914-3ebb-4b84-b359-f02af0b244d8 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:51.903869 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1e914-3ebb-4b84-b359-f02af0b244d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:51.904163 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1e914-3ebb-4b84-b359-f02af0b244d8 None admin] PATH_INFO: `/v3/projects/95973b8149ff42f7a55d58cfb84200a8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:51.905255 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f1e914-3ebb-4b84-b359-f02af0b244d8 None admin] RBAC: Authorizing `identity:get_project(project_id=95973b8149ff42f7a55d58cfb84200a8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:51.906656 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f1e914-3ebb-4b84-b359-f02af0b244d8 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:51.908239 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 574/1145] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:48:51 2022] GET /identity/v3/projects/95973b8149ff42f7a55d58cfb84200a8 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:51.914674 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90658dfd-4ec8-48c0-b3a5-fa3aaf3a89df None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:51.938371 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:52.012501 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90658dfd-4ec8-48c0-b3a5-fa3aaf3a89df None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:52.046404 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c1a06-6687-423b-a14d-a31643c29ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:52.047003 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c1a06-6687-423b-a14d-a31643c29ab9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:52.047282 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c1a06-6687-423b-a14d-a31643c29ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:52.047552 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c1a06-6687-423b-a14d-a31643c29ab9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:52.048302 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c1a06-6687-423b-a14d-a31643c29ab9 None admin] RBAC: Adding query filter params (role.id=fcbf519fbc84458f913671a5a1d0914c, scope.project.id=95973b8149ff42f7a55d58cfb84200a8, user.id=57c8811a06dd474ea802e187d1ac0299) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:52.048615 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c1a06-6687-423b-a14d-a31643c29ab9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:52.050045 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c1a06-6687-423b-a14d-a31643c29ab9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:52.055218 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 572/1146] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:48:51 2022] GET /identity/v3/role_assignments?user.id=57c8811a06dd474ea802e187d1ac0299&scope.project.id=95973b8149ff42f7a55d58cfb84200a8&role.id=fcbf519fbc84458f913671a5a1d0914c => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:54.135039 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86643478-b89d-4567-bdd7-902d2cc1f9cc None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:54.135564 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86643478-b89d-4567-bdd7-902d2cc1f9cc None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:54.135564 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86643478-b89d-4567-bdd7-902d2cc1f9cc None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:54.136926 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 575/1147] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:48:54.144251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1eacc-3bda-48f7-807d-1625c96b5b0d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:54.144704 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1eacc-3bda-48f7-807d-1625c96b5b0d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:54.144805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1eacc-3bda-48f7-807d-1625c96b5b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:54.162917 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1db1eacc-3bda-48f7-807d-1625c96b5b0d None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:54.164112 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1db1eacc-3bda-48f7-807d-1625c96b5b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:54.228798 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 573/1148] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:48:54.505831 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86643478-b89d-4567-bdd7-902d2cc1f9cc None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:54.599334 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:54.603184 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86643478-b89d-4567-bdd7-902d2cc1f9cc None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:54.635795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afabf3fe-b860-4850-88e9-758579c50771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:54.636437 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabf3fe-b860-4850-88e9-758579c50771 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:54.636715 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabf3fe-b860-4850-88e9-758579c50771 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:54.636984 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabf3fe-b860-4850-88e9-758579c50771 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:54.647143 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afabf3fe-b860-4850-88e9-758579c50771 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:48:54.651918 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-afabf3fe-b860-4850-88e9-758579c50771 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:48:54.651918 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:54.651918 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:54.651918 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:54.651918 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:54.651918 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:54.651918 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:54.651918 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:54.651918 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:54.651918 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:54.651918 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:54.651918 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:54.651918 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:48:54.651918 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:48:54.651918 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:48:54.651918 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:48:54.652953 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:48:54.652953 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:48:54.652953 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:48:54.652953 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:48:54.652953 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:54.652953 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:54.652953 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:54.652953 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:54.652953 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:54.652953 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:54.652953 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:54.652953 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:54.652953 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:54.652953 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:54.652953 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:54.652953 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:54.653905 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:54.653905 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:54.653905 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:54.653905 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:54.653905 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:48:54.653905 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:48:54.653905 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:48:54.653905 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:48:54.653905 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:48:54.653905 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:48:54.653905 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:48:54.653905 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:48:54.653905 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 576/1149] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:48:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:54.659025 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1eacc-3bda-48f7-807d-1625c96b5b0d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:54.676944 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:54.755916 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1eacc-3bda-48f7-807d-1625c96b5b0d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:54.788495 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b20da4f-fbd4-4837-9240-25fd814c379d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:54.789102 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20da4f-fbd4-4837-9240-25fd814c379d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:54.789384 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20da4f-fbd4-4837-9240-25fd814c379d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:54.789657 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20da4f-fbd4-4837-9240-25fd814c379d None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:54.790317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b20da4f-fbd4-4837-9240-25fd814c379d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:54.790620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b20da4f-fbd4-4837-9240-25fd814c379d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:54.792003 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b20da4f-fbd4-4837-9240-25fd814c379d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:54.800620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 574/1150] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:48:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:54.807740 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afabf3fe-b860-4850-88e9-758579c50771 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:54.903326 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afabf3fe-b860-4850-88e9-758579c50771 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:54.935573 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6562ad-c08b-43d9-8d53-62ab4f1017ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:54.936208 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6562ad-c08b-43d9-8d53-62ab4f1017ed None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:54.936485 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6562ad-c08b-43d9-8d53-62ab4f1017ed None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:54.936749 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6562ad-c08b-43d9-8d53-62ab4f1017ed None admin] PATH_INFO: `/v3/users/57c8811a06dd474ea802e187d1ac0299` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:54.938090 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6562ad-c08b-43d9-8d53-62ab4f1017ed None admin] RBAC: Authorizing `identity:get_user(user_id=57c8811a06dd474ea802e187d1ac0299)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:54.939488 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6562ad-c08b-43d9-8d53-62ab4f1017ed None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:54.941044 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 577/1151] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:48:54 2022] GET /identity/v3/users/57c8811a06dd474ea802e187d1ac0299 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:54.946805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b20da4f-fbd4-4837-9240-25fd814c379d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:55.023282 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:55.044235 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b20da4f-fbd4-4837-9240-25fd814c379d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:55.076391 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce3db9-72ee-4fa7-8bf3-18508170ecdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:55.077012 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce3db9-72ee-4fa7-8bf3-18508170ecdd None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:55.077291 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce3db9-72ee-4fa7-8bf3-18508170ecdd None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:55.077564 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce3db9-72ee-4fa7-8bf3-18508170ecdd None admin] PATH_INFO: `/v3/projects/95973b8149ff42f7a55d58cfb84200a8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:55.078667 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce3db9-72ee-4fa7-8bf3-18508170ecdd None admin] RBAC: Authorizing `identity:get_project(project_id=95973b8149ff42f7a55d58cfb84200a8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:55.080119 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce3db9-72ee-4fa7-8bf3-18508170ecdd None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:55.081728 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 575/1152] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:48:54 2022] GET /identity/v3/projects/95973b8149ff42f7a55d58cfb84200a8 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:55.088471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6562ad-c08b-43d9-8d53-62ab4f1017ed None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:55.117178 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:55.184514 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6562ad-c08b-43d9-8d53-62ab4f1017ed None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:55.216773 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64860bbc-8947-4a29-b0ed-47b85eb80392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:55.217417 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64860bbc-8947-4a29-b0ed-47b85eb80392 None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:55.217691 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64860bbc-8947-4a29-b0ed-47b85eb80392 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:55.217958 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64860bbc-8947-4a29-b0ed-47b85eb80392 None admin] PATH_INFO: `/v3/projects/95973b8149ff42f7a55d58cfb84200a8/users/57c8811a06dd474ea802e187d1ac0299/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:55.219798 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64860bbc-8947-4a29-b0ed-47b85eb80392 None admin] RBAC: Authorizing `identity:create_grant(project_id=95973b8149ff42f7a55d58cfb84200a8, user_id=57c8811a06dd474ea802e187d1ac0299, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:55.221240 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64860bbc-8947-4a29-b0ed-47b85eb80392 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:55.318261 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 578/1153] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Sep 9 20:48:55 2022] PUT /identity/v3/projects/95973b8149ff42f7a55d58cfb84200a8/users/57c8811a06dd474ea802e187d1ac0299/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:48:57.414946 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79420b61-faa0-4de7-93c2-9f3effae8aaa None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:57.415485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79420b61-faa0-4de7-93c2-9f3effae8aaa None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:57.415485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79420b61-faa0-4de7-93c2-9f3effae8aaa None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:57.416884 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 576/1154] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:48:57.422323 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60218142-e6d3-4835-b8ea-ce1882f3e604 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:57.422727 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60218142-e6d3-4835-b8ea-ce1882f3e604 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:57.422867 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60218142-e6d3-4835-b8ea-ce1882f3e604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:57.440834 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-60218142-e6d3-4835-b8ea-ce1882f3e604 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:48:57.442038 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-60218142-e6d3-4835-b8ea-ce1882f3e604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:48:57.504854 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 579/1155] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:48:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:48:57.772831 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79420b61-faa0-4de7-93c2-9f3effae8aaa None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:57.870227 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79420b61-faa0-4de7-93c2-9f3effae8aaa None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:57.902524 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c79c5-b2e7-4c23-8964-f2f35bef4d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:57.903134 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c79c5-b2e7-4c23-8964-f2f35bef4d7a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:57.903412 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c79c5-b2e7-4c23-8964-f2f35bef4d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:57.903683 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c79c5-b2e7-4c23-8964-f2f35bef4d7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:57.913838 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c79c5-b2e7-4c23-8964-f2f35bef4d7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:48:57.918598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-884c79c5-b2e7-4c23-8964-f2f35bef4d7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:48:57.918598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:48:57.918598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:48:57.918598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:48:57.918598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:48:57.918598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:48:57.918598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:48:57.918598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:48:57.918598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:48:57.918598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:48:57.918598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:48:57.918598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:48:57.918598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:48:57.918598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:48:57.918598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:48:57.918598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:48:57.919662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:48:57.919662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:48:57.919662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:48:57.919662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:48:57.919662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:48:57.919662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:48:57.919662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:48:57.919662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:48:57.919662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:48:57.919662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:48:57.919662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:48:57.919662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:48:57.919662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:48:57.919662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:48:57.919662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:48:57.919662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:48:57.920582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:48:57.920582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:48:57.920582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:48:57.920582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:48:57.920582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:48:57.920582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:48:57.920582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:48:57.920582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:48:57.920582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:48:57.920582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:48:57.920582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:48:57.920582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:48:57.920582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 577/1156] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:48:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:48:57.925564 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60218142-e6d3-4835-b8ea-ce1882f3e604 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:57.943915 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:58.022740 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60218142-e6d3-4835-b8ea-ce1882f3e604 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:58.055406 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae01ef-c297-4eab-aa1b-b1983e83dc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:58.055948 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae01ef-c297-4eab-aa1b-b1983e83dc2a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:58.056256 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae01ef-c297-4eab-aa1b-b1983e83dc2a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:58.056528 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae01ef-c297-4eab-aa1b-b1983e83dc2a None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:58.057191 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ae01ef-c297-4eab-aa1b-b1983e83dc2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:58.057490 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ae01ef-c297-4eab-aa1b-b1983e83dc2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:58.058833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ae01ef-c297-4eab-aa1b-b1983e83dc2a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:58.067378 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 580/1157] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:48:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:58.073234 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c79c5-b2e7-4c23-8964-f2f35bef4d7a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:58.082071 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:58.168963 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c79c5-b2e7-4c23-8964-f2f35bef4d7a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:58.201934 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b0fe1-1c3d-4b68-84f1-a8ac19392e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:58.202546 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b0fe1-1c3d-4b68-84f1-a8ac19392e25 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:58.202827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b0fe1-1c3d-4b68-84f1-a8ac19392e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:58.203099 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b0fe1-1c3d-4b68-84f1-a8ac19392e25 None admin] PATH_INFO: `/v3/users/57c8811a06dd474ea802e187d1ac0299` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:58.204226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90b0fe1-1c3d-4b68-84f1-a8ac19392e25 None admin] RBAC: Authorizing `identity:get_user(user_id=57c8811a06dd474ea802e187d1ac0299)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:58.205646 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90b0fe1-1c3d-4b68-84f1-a8ac19392e25 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:58.207180 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 578/1158] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:48:58 2022] GET /identity/v3/users/57c8811a06dd474ea802e187d1ac0299 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:48:58.212883 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae01ef-c297-4eab-aa1b-b1983e83dc2a None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:58.290634 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:58.311694 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae01ef-c297-4eab-aa1b-b1983e83dc2a None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:58.344950 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc14f84-9191-425f-81f6-b06d004cb308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:58.345547 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc14f84-9191-425f-81f6-b06d004cb308 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:58.345834 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc14f84-9191-425f-81f6-b06d004cb308 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:58.346108 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc14f84-9191-425f-81f6-b06d004cb308 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:58.410143 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc14f84-9191-425f-81f6-b06d004cb308 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:58.412145 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc14f84-9191-425f-81f6-b06d004cb308 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:58.489661 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:58.508752 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 581/1159] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:48:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:48:58.514703 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b0fe1-1c3d-4b68-84f1-a8ac19392e25 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:58.587076 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:48:58.610960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b0fe1-1c3d-4b68-84f1-a8ac19392e25 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:58.644291 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1628dda-42ea-4874-80c4-100e57d84f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:58.645030 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1628dda-42ea-4874-80c4-100e57d84f08 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:58.645310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1628dda-42ea-4874-80c4-100e57d84f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:58.645584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1628dda-42ea-4874-80c4-100e57d84f08 None admin] PATH_INFO: `/v3/projects/95973b8149ff42f7a55d58cfb84200a8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:58.646688 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1628dda-42ea-4874-80c4-100e57d84f08 None admin] RBAC: Authorizing `identity:get_project(project_id=95973b8149ff42f7a55d58cfb84200a8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:58.648146 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1628dda-42ea-4874-80c4-100e57d84f08 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:58.649732 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 579/1160] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:48:58 2022] GET /identity/v3/projects/95973b8149ff42f7a55d58cfb84200a8 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:48:58.655720 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc14f84-9191-425f-81f6-b06d004cb308 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:48:58.751267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc14f84-9191-425f-81f6-b06d004cb308 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:48:58.784282 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef636f94-cf88-45d9-b176-c7e7b913a3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:48:58.784887 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef636f94-cf88-45d9-b176-c7e7b913a3ab None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:48:58.785169 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef636f94-cf88-45d9-b176-c7e7b913a3ab None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:48:58.786119 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef636f94-cf88-45d9-b176-c7e7b913a3ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:48:58.786858 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef636f94-cf88-45d9-b176-c7e7b913a3ab None admin] RBAC: Adding query filter params (role.id=fcbf519fbc84458f913671a5a1d0914c, scope.project.id=95973b8149ff42f7a55d58cfb84200a8, user.id=57c8811a06dd474ea802e187d1ac0299) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:48:58.787163 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef636f94-cf88-45d9-b176-c7e7b913a3ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:48:58.789043 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef636f94-cf88-45d9-b176-c7e7b913a3ab None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:48:58.795100 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 582/1161] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:48:58 2022] GET /identity/v3/role_assignments?user.id=57c8811a06dd474ea802e187d1ac0299&scope.project.id=95973b8149ff42f7a55d58cfb84200a8&role.id=fcbf519fbc84458f913671a5a1d0914c => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:49:00.932668 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65003c-bf87-49de-a817-fb836a7d1b5b None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:00.933233 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65003c-bf87-49de-a817-fb836a7d1b5b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:00.933233 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65003c-bf87-49de-a817-fb836a7d1b5b None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:00.934537 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 580/1162] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:49:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:49:00.939991 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212a4fe-9af2-4e86-99cc-4d155c758fd1 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:00.940462 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212a4fe-9af2-4e86-99cc-4d155c758fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:00.940570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212a4fe-9af2-4e86-99cc-4d155c758fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:00.959348 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-5212a4fe-9af2-4e86-99cc-4d155c758fd1 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:49:00.960580 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5212a4fe-9af2-4e86-99cc-4d155c758fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:49:00.999522 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:49:01.026314 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 583/1163] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:49:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:49:01.290541 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65003c-bf87-49de-a817-fb836a7d1b5b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:49:01.389603 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65003c-bf87-49de-a817-fb836a7d1b5b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:49:01.421202 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3000b8-623b-47e8-a2ca-745b9cee4944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:49:01.421785 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3000b8-623b-47e8-a2ca-745b9cee4944 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:01.422060 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3000b8-623b-47e8-a2ca-745b9cee4944 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:01.422329 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3000b8-623b-47e8-a2ca-745b9cee4944 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:01.433914 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:49:01.491637 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3000b8-623b-47e8-a2ca-745b9cee4944 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:49:01.493102 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3000b8-623b-47e8-a2ca-745b9cee4944 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:49:01.585617 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 581/1164] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:49:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:49:01.591565 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5212a4fe-9af2-4e86-99cc-4d155c758fd1 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:49:01.686292 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5212a4fe-9af2-4e86-99cc-4d155c758fd1 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:49:01.718531 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93905a5-a7f0-490f-abae-48badb84d39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:49:01.719150 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93905a5-a7f0-490f-abae-48badb84d39c None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:01.719429 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93905a5-a7f0-490f-abae-48badb84d39c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:01.719699 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93905a5-a7f0-490f-abae-48badb84d39c None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:01.726756 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93905a5-a7f0-490f-abae-48badb84d39c None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:49:01.728194 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93905a5-a7f0-490f-abae-48badb84d39c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:49:01.739901 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-e93905a5-a7f0-490f-abae-48badb84d39c None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 09 20:49:01.739901 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:49:01.739901 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 09 20:49:01.739901 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 09 20:49:01.739901 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:49:01.739901 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:49:01.739901 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:49:01.739901 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:49:01.739901 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 09 20:49:01.739901 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:49:01.739901 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:49:01.739901 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:49:01.739901 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:49:01.739901 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:49:01.739901 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:49:01.739901 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:49:01.739901 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:49:01.740947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:49:01.740947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:49:01.740947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:49:01.740947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:49:01.740947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:49:01.740947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:49:01.740947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:49:01.740947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:49:01.740947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:49:01.740947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:49:01.740947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:49:01.740947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:49:01.740947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:49:01.740947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:49:01.740947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:49:01.740947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:49:01.741894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:49:01.741894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:49:01.741894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:49:01.741894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:49:01.741894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:49:01.741894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:49:01.741894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:49:01.741894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:49:01.741894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:49:01.741894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:49:01.741894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 09 20:49:01.741894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:49:01.741894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 09 20:49:01.741894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:49:01.741894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 584/1165] 10.222.0.48 () {56 vars in 1117 bytes} [Fri Sep 9 20:49:01 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:49:01.746877 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3000b8-623b-47e8-a2ca-745b9cee4944 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:49:01.788868 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:49:01.846083 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3000b8-623b-47e8-a2ca-745b9cee4944 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:49:01.878893 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629b62a-3c67-49ef-9bab-ee3e55b930e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:49:01.879575 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629b62a-3c67-49ef-9bab-ee3e55b930e0 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:01.879852 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629b62a-3c67-49ef-9bab-ee3e55b930e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:01.880146 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629b62a-3c67-49ef-9bab-ee3e55b930e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:01.880782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629b62a-3c67-49ef-9bab-ee3e55b930e0 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:49:01.881083 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629b62a-3c67-49ef-9bab-ee3e55b930e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:49:01.882445 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629b62a-3c67-49ef-9bab-ee3e55b930e0 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:49:01.889311 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 582/1166] 10.222.0.48 () {56 vars in 1115 bytes} [Fri Sep 9 20:49:01 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:49:03.999973 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbd166-e6e9-4dd6-b96b-26ab4fc8b3e0 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:04.000560 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbd166-e6e9-4dd6-b96b-26ab4fc8b3e0 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:04.000560 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbd166-e6e9-4dd6-b96b-26ab4fc8b3e0 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:04.001874 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 585/1167] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:49:04.007903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa447b5-596a-492d-b722-0bd5930ac9aa None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:04.008400 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa447b5-596a-492d-b722-0bd5930ac9aa None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:04.008499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa447b5-596a-492d-b722-0bd5930ac9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:04.030821 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-daa447b5-596a-492d-b722-0bd5930ac9aa None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:49:04.032005 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-daa447b5-596a-492d-b722-0bd5930ac9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:49:04.102342 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 583/1168] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:49:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:49:04.397941 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dbd166-e6e9-4dd6-b96b-26ab4fc8b3e0 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:49:04.435186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:49:04.502572 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dbd166-e6e9-4dd6-b96b-26ab4fc8b3e0 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:49:04.534284 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2eac8-ffa2-4114-878e-8eb629636f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:49:04.534880 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2eac8-ffa2-4114-878e-8eb629636f9b None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:04.535157 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2eac8-ffa2-4114-878e-8eb629636f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:04.535955 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2eac8-ffa2-4114-878e-8eb629636f9b None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:04.536641 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2eac8-ffa2-4114-878e-8eb629636f9b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:49:04.537993 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2eac8-ffa2-4114-878e-8eb629636f9b None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:49:04.639473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 586/1169] 10.222.0.48 () {60 vars in 1138 bytes} [Fri Sep 9 20:49:04 2022] POST /identity/v3/domains => generated 267 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:49:06.768563 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1808f0-bcfa-4f59-9630-2659d55c3aad None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:06.769101 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1808f0-bcfa-4f59-9630-2659d55c3aad None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:06.769101 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1808f0-bcfa-4f59-9630-2659d55c3aad None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:06.770439 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 584/1170] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:49:06.775729 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce96396-b332-4789-a13a-803a7539b3a6 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:06.776170 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce96396-b332-4789-a13a-803a7539b3a6 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:06.776311 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce96396-b332-4789-a13a-803a7539b3a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:06.795241 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-1ce96396-b332-4789-a13a-803a7539b3a6 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:49:06.796447 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-1ce96396-b332-4789-a13a-803a7539b3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:49:06.859141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 587/1171] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:49:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:49:07.126002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1808f0-bcfa-4f59-9630-2659d55c3aad None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:49:07.161631 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:49:07.224602 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1808f0-bcfa-4f59-9630-2659d55c3aad None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:49:07.256583 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f1500-e677-4c02-b4a3-8b04ebe298c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:49:07.257199 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f1500-e677-4c02-b4a3-8b04ebe298c5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:07.257485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f1500-e677-4c02-b4a3-8b04ebe298c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:07.257756 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f1500-e677-4c02-b4a3-8b04ebe298c5 None admin] PATH_INFO: `/v3/domains/1c9f899c026e42aeabbc3f962f9a6bb3` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:07.267691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f1500-e677-4c02-b4a3-8b04ebe298c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1c9f899c026e42aeabbc3f962f9a6bb3)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:49:07.269167 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f1500-e677-4c02-b4a3-8b04ebe298c5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:49:07.271247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 585/1172] 10.222.0.48 () {56 vars in 1183 bytes} [Fri Sep 9 20:49:07 2022] GET /identity/v3/domains/1c9f899c026e42aeabbc3f962f9a6bb3 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:49:07.277009 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce96396-b332-4789-a13a-803a7539b3a6 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:49:07.301018 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:49:07.371731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce96396-b332-4789-a13a-803a7539b3a6 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:49:07.404839 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344afcfe-8a5c-4a13-9fd2-db68ca307083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:49:07.405434 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344afcfe-8a5c-4a13-9fd2-db68ca307083 None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:07.405711 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344afcfe-8a5c-4a13-9fd2-db68ca307083 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:07.405982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344afcfe-8a5c-4a13-9fd2-db68ca307083 None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:07.406629 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344afcfe-8a5c-4a13-9fd2-db68ca307083 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:49:07.408044 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344afcfe-8a5c-4a13-9fd2-db68ca307083 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:49:07.479073 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 588/1173] 10.222.0.48 () {60 vars in 1141 bytes} [Fri Sep 9 20:49:07 2022] POST /identity/v3/projects => generated 361 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:49:09.620453 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7749d2-53b6-4840-90bc-27f20d8ebc37 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:09.621021 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7749d2-53b6-4840-90bc-27f20d8ebc37 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:09.621021 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7749d2-53b6-4840-90bc-27f20d8ebc37 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:09.622331 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 586/1174] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:49:09.627455 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cea29-57c0-4057-8365-62ff1c351771 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:09.627840 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cea29-57c0-4057-8365-62ff1c351771 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:09.628007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cea29-57c0-4057-8365-62ff1c351771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:09.646926 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-da8cea29-57c0-4057-8365-62ff1c351771 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:49:09.648121 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-da8cea29-57c0-4057-8365-62ff1c351771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:49:09.711750 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 589/1175] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:49:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:49:09.980313 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7749d2-53b6-4840-90bc-27f20d8ebc37 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:49:10.074153 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:49:10.080451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7749d2-53b6-4840-90bc-27f20d8ebc37 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:49:10.112028 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf3533-9737-4414-b81d-d4661057de3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:49:10.112680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf3533-9737-4414-b81d-d4661057de3e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:10.112958 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf3533-9737-4414-b81d-d4661057de3e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:10.113744 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf3533-9737-4414-b81d-d4661057de3e None admin] PATH_INFO: `/v3/domains/1c9f899c026e42aeabbc3f962f9a6bb3` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:10.114847 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf3533-9737-4414-b81d-d4661057de3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=1c9f899c026e42aeabbc3f962f9a6bb3)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:49:10.116301 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf3533-9737-4414-b81d-d4661057de3e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:49:10.117848 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 587/1176] 10.222.0.48 () {56 vars in 1183 bytes} [Fri Sep 9 20:49:09 2022] GET /identity/v3/domains/1c9f899c026e42aeabbc3f962f9a6bb3 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:49:10.124865 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8cea29-57c0-4057-8365-62ff1c351771 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:49:10.135528 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:49:10.225368 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8cea29-57c0-4057-8365-62ff1c351771 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:49:10.258804 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b2545-3b83-4aa1-842e-4325a4170e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:49:10.259374 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b2545-3b83-4aa1-842e-4325a4170e96 None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:10.259652 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b2545-3b83-4aa1-842e-4325a4170e96 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:10.259920 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b2545-3b83-4aa1-842e-4325a4170e96 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:10.260605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b2545-3b83-4aa1-842e-4325a4170e96 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:49:10.262017 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b2545-3b83-4aa1-842e-4325a4170e96 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:49:10.352847 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 590/1177] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Sep 9 20:49:10 2022] POST /identity/v3/users => generated 311 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:49:12.541439 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a46cb3-e6af-41fe-8803-cf702a14944d None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:12.542010 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a46cb3-e6af-41fe-8803-cf702a14944d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:12.542010 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a46cb3-e6af-41fe-8803-cf702a14944d None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:12.543313 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 588/1178] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:49:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:49:12.548498 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f0e3e-b58f-447b-85eb-2f4d24669dd3 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:12.548884 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f0e3e-b58f-447b-85eb-2f4d24669dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:12.549055 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f0e3e-b58f-447b-85eb-2f4d24669dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:12.567022 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-224f0e3e-b58f-447b-85eb-2f4d24669dd3 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:49:12.568249 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-224f0e3e-b58f-447b-85eb-2f4d24669dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:49:12.624113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:49:12.634677 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 591/1179] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:49:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:49:12.905251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a46cb3-e6af-41fe-8803-cf702a14944d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:49:13.003386 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a46cb3-e6af-41fe-8803-cf702a14944d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:49:13.036806 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b11681-e7a7-4666-969d-df49ec45909f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:49:13.037429 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b11681-e7a7-4666-969d-df49ec45909f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:13.037706 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b11681-e7a7-4666-969d-df49ec45909f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:13.037972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b11681-e7a7-4666-969d-df49ec45909f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:13.047718 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b11681-e7a7-4666-969d-df49ec45909f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:49:13.053241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-62b11681-e7a7-4666-969d-df49ec45909f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:49:13.053241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:49:13.053241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:49:13.053241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:49:13.053241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:49:13.053241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:49:13.053241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:49:13.053241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:49:13.053241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:49:13.053241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:49:13.053241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:49:13.053241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:49:13.053241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:49:13.053241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:49:13.053241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:49:13.053241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:49:13.054402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:49:13.054402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:49:13.054402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:49:13.054402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:49:13.054402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:49:13.054402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:49:13.054402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:49:13.054402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:49:13.054402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:49:13.054402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:49:13.054402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:49:13.054402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:49:13.054402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:49:13.054402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:49:13.054402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:49:13.054402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:49:13.055482 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:49:13.055482 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:49:13.055482 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:49:13.055482 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:49:13.055482 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:49:13.055482 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:49:13.055482 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:49:13.055482 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:49:13.055482 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:49:13.055482 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:49:13.055482 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:49:13.055482 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:49:13.055482 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 589/1180] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:49:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:49:13.060367 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f0e3e-b58f-447b-85eb-2f4d24669dd3 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:49:13.155429 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f0e3e-b58f-447b-85eb-2f4d24669dd3 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:49:13.188298 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f9022-4b97-46cb-91b4-b474c72d641d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:49:13.188893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f9022-4b97-46cb-91b4-b474c72d641d None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:13.189174 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f9022-4b97-46cb-91b4-b474c72d641d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:13.189448 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f9022-4b97-46cb-91b4-b474c72d641d None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:13.190110 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f9022-4b97-46cb-91b4-b474c72d641d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:49:13.190415 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f9022-4b97-46cb-91b4-b474c72d641d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:49:13.192394 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f9022-4b97-46cb-91b4-b474c72d641d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:49:13.200063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 592/1181] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:49:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:49:13.206013 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b11681-e7a7-4666-969d-df49ec45909f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:49:13.230576 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:49:13.301965 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b11681-e7a7-4666-969d-df49ec45909f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:49:13.334465 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae9c6e-e84c-492a-ac0a-c7ec1bb832b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:49:13.335069 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae9c6e-e84c-492a-ac0a-c7ec1bb832b5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:13.335342 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae9c6e-e84c-492a-ac0a-c7ec1bb832b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:13.335611 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae9c6e-e84c-492a-ac0a-c7ec1bb832b5 None admin] PATH_INFO: `/v3/users/fe42e69073b04a3b9620cd9f4794101b` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:13.347896 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae9c6e-e84c-492a-ac0a-c7ec1bb832b5 None admin] RBAC: Authorizing `identity:get_user(user_id=fe42e69073b04a3b9620cd9f4794101b)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:49:13.349360 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae9c6e-e84c-492a-ac0a-c7ec1bb832b5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:49:13.350904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 590/1182] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:49:13 2022] GET /identity/v3/users/fe42e69073b04a3b9620cd9f4794101b => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:49:13.356559 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f9022-4b97-46cb-91b4-b474c72d641d None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:49:13.409936 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:49:13.452790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f9022-4b97-46cb-91b4-b474c72d641d None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:49:13.484834 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78bd2b2-d28f-41a1-859b-fc325dc67f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:49:13.485425 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78bd2b2-d28f-41a1-859b-fc325dc67f0f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:13.485700 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78bd2b2-d28f-41a1-859b-fc325dc67f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:13.485966 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78bd2b2-d28f-41a1-859b-fc325dc67f0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:13.549791 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78bd2b2-d28f-41a1-859b-fc325dc67f0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:49:13.551792 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78bd2b2-d28f-41a1-859b-fc325dc67f0f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:49:13.602160 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:49:13.645491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 593/1183] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:49:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:49:13.651245 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae9c6e-e84c-492a-ac0a-c7ec1bb832b5 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:49:13.729232 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:49:13.746777 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae9c6e-e84c-492a-ac0a-c7ec1bb832b5 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:49:13.789820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4de0d6-0f26-493e-b734-61bcb55463c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:49:13.790450 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4de0d6-0f26-493e-b734-61bcb55463c5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:13.790732 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4de0d6-0f26-493e-b734-61bcb55463c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:13.791005 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4de0d6-0f26-493e-b734-61bcb55463c5 None admin] PATH_INFO: `/v3/projects/25a368f94f824294b3e2e46d80cc3f08` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:13.792144 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4de0d6-0f26-493e-b734-61bcb55463c5 None admin] RBAC: Authorizing `identity:get_project(project_id=25a368f94f824294b3e2e46d80cc3f08)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:49:13.793585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4de0d6-0f26-493e-b734-61bcb55463c5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:49:13.795206 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 591/1184] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:49:13 2022] GET /identity/v3/projects/25a368f94f824294b3e2e46d80cc3f08 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:49:13.801316 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78bd2b2-d28f-41a1-859b-fc325dc67f0f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:49:13.897074 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78bd2b2-d28f-41a1-859b-fc325dc67f0f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:49:13.929815 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a2b68-1bf2-489d-95cd-73ca4680b57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:49:13.930410 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a2b68-1bf2-489d-95cd-73ca4680b57b None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:13.930688 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a2b68-1bf2-489d-95cd-73ca4680b57b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:13.930959 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a2b68-1bf2-489d-95cd-73ca4680b57b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:13.931674 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41a2b68-1bf2-489d-95cd-73ca4680b57b None admin] RBAC: Adding query filter params (role.id=fcbf519fbc84458f913671a5a1d0914c, scope.project.id=25a368f94f824294b3e2e46d80cc3f08, user.id=fe42e69073b04a3b9620cd9f4794101b) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:49:13.931979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41a2b68-1bf2-489d-95cd-73ca4680b57b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:49:13.933436 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41a2b68-1bf2-489d-95cd-73ca4680b57b None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:49:13.938816 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 594/1185] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:49:13 2022] GET /identity/v3/role_assignments?user.id=fe42e69073b04a3b9620cd9f4794101b&scope.project.id=25a368f94f824294b3e2e46d80cc3f08&role.id=fcbf519fbc84458f913671a5a1d0914c => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:49:16.055924 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a27d8-a742-4d88-b279-d41f20f3fc0e None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:16.056513 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a27d8-a742-4d88-b279-d41f20f3fc0e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:16.056513 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a27d8-a742-4d88-b279-d41f20f3fc0e None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:16.057843 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 592/1186] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:49:16.063270 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466a6a0-bdcb-42de-9f6c-0723f25550c7 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:16.063686 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466a6a0-bdcb-42de-9f6c-0723f25550c7 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:16.063813 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466a6a0-bdcb-42de-9f6c-0723f25550c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:16.082215 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-5466a6a0-bdcb-42de-9f6c-0723f25550c7 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:49:16.083412 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5466a6a0-bdcb-42de-9f6c-0723f25550c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:49:16.096251 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:49:16.147361 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 595/1187] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:49:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:49:16.421632 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a27d8-a742-4d88-b279-d41f20f3fc0e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:49:16.518896 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a27d8-a742-4d88-b279-d41f20f3fc0e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:49:16.551562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044eaba0-9320-45d2-9227-3c6a9b2df42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:49:16.552193 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044eaba0-9320-45d2-9227-3c6a9b2df42e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:16.552469 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044eaba0-9320-45d2-9227-3c6a9b2df42e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:16.552733 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044eaba0-9320-45d2-9227-3c6a9b2df42e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:16.562264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044eaba0-9320-45d2-9227-3c6a9b2df42e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:49:16.567241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-044eaba0-9320-45d2-9227-3c6a9b2df42e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:49:16.567241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:49:16.567241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:49:16.567241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:49:16.567241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:49:16.567241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:49:16.567241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:49:16.567241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:49:16.567241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:49:16.567241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:49:16.567241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:49:16.567241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:49:16.567241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:49:16.567241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:49:16.567241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:49:16.567241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:49:16.568382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:49:16.568382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:49:16.568382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:49:16.568382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:49:16.568382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:49:16.568382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:49:16.568382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:49:16.568382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:49:16.568382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:49:16.568382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:49:16.568382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:49:16.568382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:49:16.568382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:49:16.568382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:49:16.568382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:49:16.568382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:49:16.569241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:49:16.569241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:49:16.569241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:49:16.569241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:49:16.569241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:49:16.569241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:49:16.569241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:49:16.569241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:49:16.569241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:49:16.569241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:49:16.569241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:49:16.569241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:49:16.569241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 593/1188] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:49:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:49:16.574237 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5466a6a0-bdcb-42de-9f6c-0723f25550c7 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:49:16.668749 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5466a6a0-bdcb-42de-9f6c-0723f25550c7 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:49:16.701238 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318bf0f-4944-4c39-b70d-e4f88a35ed6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:49:16.701767 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318bf0f-4944-4c39-b70d-e4f88a35ed6d None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:16.702042 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318bf0f-4944-4c39-b70d-e4f88a35ed6d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:16.702307 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318bf0f-4944-4c39-b70d-e4f88a35ed6d None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:16.702955 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318bf0f-4944-4c39-b70d-e4f88a35ed6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:49:16.703250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318bf0f-4944-4c39-b70d-e4f88a35ed6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:49:16.704622 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318bf0f-4944-4c39-b70d-e4f88a35ed6d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:49:16.712166 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 596/1189] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:49:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:49:16.717913 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044eaba0-9320-45d2-9227-3c6a9b2df42e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:49:16.731479 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:49:16.815647 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044eaba0-9320-45d2-9227-3c6a9b2df42e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:49:16.848461 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a227897-cd79-4bfc-8a75-46afdbad1d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:49:16.849068 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a227897-cd79-4bfc-8a75-46afdbad1d35 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:16.849383 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a227897-cd79-4bfc-8a75-46afdbad1d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:16.849642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a227897-cd79-4bfc-8a75-46afdbad1d35 None admin] PATH_INFO: `/v3/users/fe42e69073b04a3b9620cd9f4794101b` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:16.850748 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a227897-cd79-4bfc-8a75-46afdbad1d35 None admin] RBAC: Authorizing `identity:get_user(user_id=fe42e69073b04a3b9620cd9f4794101b)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:49:16.852203 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a227897-cd79-4bfc-8a75-46afdbad1d35 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:49:16.853742 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 594/1190] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:49:16 2022] GET /identity/v3/users/fe42e69073b04a3b9620cd9f4794101b => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:49:16.859436 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318bf0f-4944-4c39-b70d-e4f88a35ed6d None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:49:16.871597 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:49:16.954491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318bf0f-4944-4c39-b70d-e4f88a35ed6d None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:49:16.986859 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ca916-b330-45bd-b78f-ea1061c72896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:49:16.987502 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ca916-b330-45bd-b78f-ea1061c72896 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:16.987776 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ca916-b330-45bd-b78f-ea1061c72896 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:16.988042 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ca916-b330-45bd-b78f-ea1061c72896 None admin] PATH_INFO: `/v3/projects/25a368f94f824294b3e2e46d80cc3f08` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:16.989218 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ca916-b330-45bd-b78f-ea1061c72896 None admin] RBAC: Authorizing `identity:get_project(project_id=25a368f94f824294b3e2e46d80cc3f08)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:49:16.990625 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ca916-b330-45bd-b78f-ea1061c72896 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:49:16.992234 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 597/1191] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:49:16 2022] GET /identity/v3/projects/25a368f94f824294b3e2e46d80cc3f08 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:49:16.998072 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a227897-cd79-4bfc-8a75-46afdbad1d35 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:49:17.074107 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:49:17.095119 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a227897-cd79-4bfc-8a75-46afdbad1d35 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:49:17.129746 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afcf3ed-c289-4239-a439-c1b5629269cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:49:17.130395 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afcf3ed-c289-4239-a439-c1b5629269cf None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:17.130674 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afcf3ed-c289-4239-a439-c1b5629269cf None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:17.130946 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afcf3ed-c289-4239-a439-c1b5629269cf None admin] PATH_INFO: `/v3/projects/25a368f94f824294b3e2e46d80cc3f08/users/fe42e69073b04a3b9620cd9f4794101b/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:17.132862 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afcf3ed-c289-4239-a439-c1b5629269cf None admin] RBAC: Authorizing `identity:create_grant(project_id=25a368f94f824294b3e2e46d80cc3f08, user_id=fe42e69073b04a3b9620cd9f4794101b, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:49:17.134295 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afcf3ed-c289-4239-a439-c1b5629269cf None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:49:17.220784 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 595/1192] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Sep 9 20:49:16 2022] PUT /identity/v3/projects/25a368f94f824294b3e2e46d80cc3f08/users/fe42e69073b04a3b9620cd9f4794101b/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:49:19.334563 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89a4da-1a96-4d76-acce-a3043659ab25 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:19.335125 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89a4da-1a96-4d76-acce-a3043659ab25 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:19.335125 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89a4da-1a96-4d76-acce-a3043659ab25 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:19.336370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 598/1193] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:49:19.341450 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9a069-f32b-4c90-945e-51b51485d483 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:19.341888 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9a069-f32b-4c90-945e-51b51485d483 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:19.341980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9a069-f32b-4c90-945e-51b51485d483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:19.359536 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e0b9a069-f32b-4c90-945e-51b51485d483 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:49:19.360737 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e0b9a069-f32b-4c90-945e-51b51485d483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:49:19.424894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 596/1194] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:49:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:49:19.691296 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89a4da-1a96-4d76-acce-a3043659ab25 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:49:19.767447 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:49:19.789297 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89a4da-1a96-4d76-acce-a3043659ab25 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:49:19.821379 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfd20fc-a91d-47f7-a7da-f9e727ae7da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:49:19.822987 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd20fc-a91d-47f7-a7da-f9e727ae7da4 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:19.822987 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd20fc-a91d-47f7-a7da-f9e727ae7da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:19.822987 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd20fc-a91d-47f7-a7da-f9e727ae7da4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:19.834630 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfd20fc-a91d-47f7-a7da-f9e727ae7da4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:49:19.839366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-cbfd20fc-a91d-47f7-a7da-f9e727ae7da4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:49:19.839366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:49:19.839366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:49:19.839366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:49:19.839366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:49:19.839366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:49:19.839366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:49:19.839366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:49:19.839366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:49:19.839366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:49:19.839366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:49:19.839366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:49:19.839366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:49:19.839366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:49:19.839366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:49:19.839366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:49:19.840491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:49:19.840491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:49:19.840491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:49:19.840491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:49:19.840491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:49:19.840491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:49:19.840491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:49:19.840491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:49:19.840491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:49:19.840491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:49:19.840491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:49:19.840491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:49:19.840491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:49:19.840491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:49:19.840491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:49:19.840491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:49:19.841655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:49:19.841655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:49:19.841655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:49:19.841655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:49:19.841655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:49:19.841655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:49:19.841655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:49:19.841655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:49:19.841655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:49:19.841655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:49:19.841655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:49:19.841655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:49:19.841655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 599/1195] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:49:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:49:19.846465 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9a069-f32b-4c90-945e-51b51485d483 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:49:19.910927 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:49:19.942850 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9a069-f32b-4c90-945e-51b51485d483 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:49:19.976420 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ade0b7-d9aa-4b4d-b7a5-03dde50d3bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:49:19.977017 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ade0b7-d9aa-4b4d-b7a5-03dde50d3bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:19.977299 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ade0b7-d9aa-4b4d-b7a5-03dde50d3bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:19.977580 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ade0b7-d9aa-4b4d-b7a5-03dde50d3bc6 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:19.978252 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ade0b7-d9aa-4b4d-b7a5-03dde50d3bc6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:49:19.978560 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ade0b7-d9aa-4b4d-b7a5-03dde50d3bc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:49:19.979939 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ade0b7-d9aa-4b4d-b7a5-03dde50d3bc6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:49:19.988004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 597/1196] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:49:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:49:19.993745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfd20fc-a91d-47f7-a7da-f9e727ae7da4 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:49:20.090242 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfd20fc-a91d-47f7-a7da-f9e727ae7da4 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:49:20.122394 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeabf90b-d3e7-4c3b-bd9d-16c0d0cbb49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:49:20.123009 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeabf90b-d3e7-4c3b-bd9d-16c0d0cbb49c None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:20.123287 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeabf90b-d3e7-4c3b-bd9d-16c0d0cbb49c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:20.123558 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeabf90b-d3e7-4c3b-bd9d-16c0d0cbb49c None admin] PATH_INFO: `/v3/users/fe42e69073b04a3b9620cd9f4794101b` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:20.124667 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeabf90b-d3e7-4c3b-bd9d-16c0d0cbb49c None admin] RBAC: Authorizing `identity:get_user(user_id=fe42e69073b04a3b9620cd9f4794101b)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:49:20.126085 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeabf90b-d3e7-4c3b-bd9d-16c0d0cbb49c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:49:20.127633 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 600/1197] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Sep 9 20:49:19 2022] GET /identity/v3/users/fe42e69073b04a3b9620cd9f4794101b => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:49:20.133955 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ade0b7-d9aa-4b4d-b7a5-03dde50d3bc6 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:49:20.230757 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ade0b7-d9aa-4b4d-b7a5-03dde50d3bc6 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:49:20.257848 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:49:20.263349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0b654-7576-471a-bc62-13e52ea4e35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:49:20.263946 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0b654-7576-471a-bc62-13e52ea4e35a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:20.264331 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0b654-7576-471a-bc62-13e52ea4e35a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:20.264604 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0b654-7576-471a-bc62-13e52ea4e35a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:20.328599 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0b654-7576-471a-bc62-13e52ea4e35a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:49:20.330040 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0b654-7576-471a-bc62-13e52ea4e35a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:49:20.428714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 598/1198] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:49:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:49:20.434586 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeabf90b-d3e7-4c3b-bd9d-16c0d0cbb49c None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:49:20.444395 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:49:20.532128 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeabf90b-d3e7-4c3b-bd9d-16c0d0cbb49c None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:49:20.564331 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459679dc-c02a-4f8c-8c0a-e9b44d91d257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:49:20.564946 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459679dc-c02a-4f8c-8c0a-e9b44d91d257 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:20.565225 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459679dc-c02a-4f8c-8c0a-e9b44d91d257 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:20.565496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459679dc-c02a-4f8c-8c0a-e9b44d91d257 None admin] PATH_INFO: `/v3/projects/25a368f94f824294b3e2e46d80cc3f08` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:20.567178 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459679dc-c02a-4f8c-8c0a-e9b44d91d257 None admin] RBAC: Authorizing `identity:get_project(project_id=25a368f94f824294b3e2e46d80cc3f08)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:49:20.568635 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459679dc-c02a-4f8c-8c0a-e9b44d91d257 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:49:20.570228 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 601/1199] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Sep 9 20:49:20 2022] GET /identity/v3/projects/25a368f94f824294b3e2e46d80cc3f08 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:49:20.576215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0b654-7576-471a-bc62-13e52ea4e35a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:49:20.604279 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:49:20.671296 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0b654-7576-471a-bc62-13e52ea4e35a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:49:20.705223 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b521e9-9c82-49a0-967e-7a604b684ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:49:20.705814 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b521e9-9c82-49a0-967e-7a604b684ade None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:49:20.706095 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b521e9-9c82-49a0-967e-7a604b684ade None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:49:20.706366 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b521e9-9c82-49a0-967e-7a604b684ade None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:49:20.707115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b521e9-9c82-49a0-967e-7a604b684ade None admin] RBAC: Adding query filter params (role.id=fcbf519fbc84458f913671a5a1d0914c, scope.project.id=25a368f94f824294b3e2e46d80cc3f08, user.id=fe42e69073b04a3b9620cd9f4794101b) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:49:20.707425 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b521e9-9c82-49a0-967e-7a604b684ade None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:49:20.708875 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b521e9-9c82-49a0-967e-7a604b684ade None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:49:20.714476 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 599/1200] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:49:20 2022] GET /identity/v3/role_assignments?user.id=fe42e69073b04a3b9620cd9f4794101b&scope.project.id=25a368f94f824294b3e2e46d80cc3f08&role.id=fcbf519fbc84458f913671a5a1d0914c => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:49:30.559931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=63 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:49:30.711935 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=53 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:31.646289 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4a6c3-9e63-4bcf-8a88-0bf858d07a03 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:31.646894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4a6c3-9e63-4bcf-8a88-0bf858d07a03 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:31.646894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4a6c3-9e63-4bcf-8a88-0bf858d07a03 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:31.648225 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 602/1201] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:50:31.653533 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61d205-1f20-4c6b-963b-53df55baef70 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:31.657132 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61d205-1f20-4c6b-963b-53df55baef70 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:31.657132 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61d205-1f20-4c6b-963b-53df55baef70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:31.672262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-bd61d205-1f20-4c6b-963b-53df55baef70 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:50:31.673427 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bd61d205-1f20-4c6b-963b-53df55baef70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:50:31.739140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 600/1202] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:50:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:50:32.008486 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4a6c3-9e63-4bcf-8a88-0bf858d07a03 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:32.108278 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4a6c3-9e63-4bcf-8a88-0bf858d07a03 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:32.141096 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d5aea-3c52-4b77-95af-9d82d6708993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:32.141710 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d5aea-3c52-4b77-95af-9d82d6708993 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:32.141984 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d5aea-3c52-4b77-95af-9d82d6708993 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:32.142249 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d5aea-3c52-4b77-95af-9d82d6708993 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:32.152478 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d5aea-3c52-4b77-95af-9d82d6708993 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:32.153900 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d5aea-3c52-4b77-95af-9d82d6708993 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:32.167976 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-e91d5aea-3c52-4b77-95af-9d82d6708993 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:50:32.167976 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:50:32.167976 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:50:32.167976 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:50:32.167976 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:50:32.167976 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:50:32.167976 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:50:32.167976 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:50:32.167976 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:50:32.167976 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:50:32.167976 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:50:32.167976 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:50:32.167976 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:50:32.167976 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:50:32.167976 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:50:32.167976 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:50:32.169205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:50:32.169205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:50:32.169205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:50:32.169205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:50:32.169205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:50:32.169205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:50:32.169205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:50:32.169205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:50:32.169205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:50:32.169205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:50:32.169205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:50:32.169205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:50:32.169205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:50:32.169205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:50:32.169205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:50:32.169205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:50:32.169205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:50:32.170328 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:50:32.170328 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:50:32.170328 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:50:32.170328 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 603/1203] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:50:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 20:50:32.175398 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61d205-1f20-4c6b-963b-53df55baef70 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:32.273584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61d205-1f20-4c6b-963b-53df55baef70 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:32.301336 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:32.306988 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6f4fc-d6c8-4b83-9b06-09a0cb286fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:32.307597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6f4fc-d6c8-4b83-9b06-09a0cb286fd3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:32.307870 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6f4fc-d6c8-4b83-9b06-09a0cb286fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:32.308191 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6f4fc-d6c8-4b83-9b06-09a0cb286fd3 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:32.308833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d6f4fc-d6c8-4b83-9b06-09a0cb286fd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:32.309140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d6f4fc-d6c8-4b83-9b06-09a0cb286fd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:32.310501 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d6f4fc-d6c8-4b83-9b06-09a0cb286fd3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:32.322277 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 601/1204] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:50:32.327873 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d5aea-3c52-4b77-95af-9d82d6708993 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:32.379503 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:32.423463 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d5aea-3c52-4b77-95af-9d82d6708993 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:32.455846 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf44ae2-f323-4301-85bf-55ae155fcc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:32.457050 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf44ae2-f323-4301-85bf-55ae155fcc2c None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:32.457324 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf44ae2-f323-4301-85bf-55ae155fcc2c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:32.457589 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf44ae2-f323-4301-85bf-55ae155fcc2c None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:32.458225 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf44ae2-f323-4301-85bf-55ae155fcc2c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:32.459640 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf44ae2-f323-4301-85bf-55ae155fcc2c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:32.526539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 604/1205] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Sep 9 20:50:32 2022] POST /identity/v3/users => generated 251 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:50:34.713840 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded9b07-f00b-498e-93bd-d2c52b027090 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:34.714431 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded9b07-f00b-498e-93bd-d2c52b027090 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:34.714431 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded9b07-f00b-498e-93bd-d2c52b027090 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:34.715707 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 602/1206] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:50:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:50:34.721153 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e1d8a-516c-4226-ac11-8a745caed38e None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:34.722433 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e1d8a-516c-4226-ac11-8a745caed38e None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:34.722433 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e1d8a-516c-4226-ac11-8a745caed38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:34.742993 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-7b9e1d8a-516c-4226-ac11-8a745caed38e None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:50:34.744214 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-7b9e1d8a-516c-4226-ac11-8a745caed38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:50:34.809069 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 605/1207] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:50:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:50:35.074385 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded9b07-f00b-498e-93bd-d2c52b027090 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:35.173070 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded9b07-f00b-498e-93bd-d2c52b027090 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:35.210004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04831c55-9c31-42d1-854b-0309dd0c39b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:35.210637 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04831c55-9c31-42d1-854b-0309dd0c39b7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:35.210918 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04831c55-9c31-42d1-854b-0309dd0c39b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:35.211191 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04831c55-9c31-42d1-854b-0309dd0c39b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:35.221737 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04831c55-9c31-42d1-854b-0309dd0c39b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:50:35.226643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-04831c55-9c31-42d1-854b-0309dd0c39b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:50:35.226643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:50:35.226643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:50:35.226643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:50:35.226643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:50:35.226643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:50:35.226643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:50:35.226643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:50:35.226643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:50:35.226643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:50:35.226643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:50:35.226643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:50:35.226643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:50:35.226643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:50:35.226643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:50:35.226643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:50:35.232247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:50:35.232247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:50:35.232247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:50:35.232247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:50:35.232247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:50:35.232247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:50:35.232247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:50:35.232247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:50:35.232247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:50:35.232247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:50:35.232247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:50:35.232247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:50:35.232247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:50:35.232247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:50:35.232247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:50:35.232247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:50:35.233412 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:50:35.233412 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:50:35.233412 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:50:35.233412 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:50:35.233412 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:50:35.233412 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:50:35.233412 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:50:35.233412 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:50:35.233412 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:50:35.233412 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:50:35.233412 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:50:35.233412 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:50:35.233412 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 603/1208] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Sep 9 20:50:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:50:35.234388 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e1d8a-516c-4226-ac11-8a745caed38e None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:35.274350 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:35.331891 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e1d8a-516c-4226-ac11-8a745caed38e None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:35.365543 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c017551-5485-4437-8410-242511d09e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:35.366148 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c017551-5485-4437-8410-242511d09e36 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:35.366425 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c017551-5485-4437-8410-242511d09e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:35.366697 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c017551-5485-4437-8410-242511d09e36 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:35.367358 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c017551-5485-4437-8410-242511d09e36 None admin] RBAC: Adding query filter params (name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:35.367660 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c017551-5485-4437-8410-242511d09e36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:35.369128 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c017551-5485-4437-8410-242511d09e36 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:35.376966 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 606/1209] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Sep 9 20:50:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:50:35.383297 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04831c55-9c31-42d1-854b-0309dd0c39b7 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:35.433055 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:35.482336 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04831c55-9c31-42d1-854b-0309dd0c39b7 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:35.518749 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30adc6ae-76bf-4af2-ae8c-1c43db5b0018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:35.519376 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30adc6ae-76bf-4af2-ae8c-1c43db5b0018 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:35.519649 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30adc6ae-76bf-4af2-ae8c-1c43db5b0018 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:35.519916 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30adc6ae-76bf-4af2-ae8c-1c43db5b0018 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:35.530621 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30adc6ae-76bf-4af2-ae8c-1c43db5b0018 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:35.531987 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30adc6ae-76bf-4af2-ae8c-1c43db5b0018 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:35.545869 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-30adc6ae-76bf-4af2-ae8c-1c43db5b0018 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:50:35.545869 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:50:35.545869 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:50:35.545869 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:50:35.545869 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:50:35.545869 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:50:35.545869 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:50:35.545869 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:50:35.545869 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:50:35.545869 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:50:35.545869 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:50:35.545869 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:50:35.545869 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:50:35.545869 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:50:35.545869 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:50:35.545869 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:50:35.547435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:50:35.547435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:50:35.547435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:50:35.547435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:50:35.547435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:50:35.547435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:50:35.547435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:50:35.547435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:50:35.547435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:50:35.547435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:50:35.547435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:50:35.547435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:50:35.547435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:50:35.547435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:50:35.547435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:50:35.547435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:50:35.547435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:50:35.548594 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:50:35.548594 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:50:35.548594 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:50:35.548594 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 604/1210] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:50:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:50:35.553551 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c017551-5485-4437-8410-242511d09e36 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:35.651002 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:35.653743 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c017551-5485-4437-8410-242511d09e36 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:35.686842 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7950d-2580-45b8-a268-46e16356be19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:35.687445 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7950d-2580-45b8-a268-46e16356be19 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:35.687726 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7950d-2580-45b8-a268-46e16356be19 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:35.688062 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7950d-2580-45b8-a268-46e16356be19 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:35.688729 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e7950d-2580-45b8-a268-46e16356be19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:35.689033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e7950d-2580-45b8-a268-46e16356be19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:35.691384 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e7950d-2580-45b8-a268-46e16356be19 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:35.702357 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 607/1211] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:50:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:50:35.708228 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30adc6ae-76bf-4af2-ae8c-1c43db5b0018 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:35.817573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30adc6ae-76bf-4af2-ae8c-1c43db5b0018 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:35.819064 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:35.851411 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3f01f-5166-43b4-9fa7-a3d574cac664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:35.852035 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3f01f-5166-43b4-9fa7-a3d574cac664 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:35.852349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3f01f-5166-43b4-9fa7-a3d574cac664 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:35.852654 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3f01f-5166-43b4-9fa7-a3d574cac664 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:35.859935 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3f01f-5166-43b4-9fa7-a3d574cac664 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:35.861407 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3f01f-5166-43b4-9fa7-a3d574cac664 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:35.873255 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-aad3f01f-5166-43b4-9fa7-a3d574cac664 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 20:50:35.873255 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:50:35.873255 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:50:35.873255 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:50:35.873255 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:50:35.873255 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:50:35.873255 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:50:35.873255 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:50:35.873255 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:50:35.873255 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:50:35.873255 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:50:35.873255 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:50:35.873255 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:50:35.873255 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:50:35.873255 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:50:35.873255 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:50:35.874382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:50:35.874382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:50:35.874382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:50:35.874382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:50:35.874382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:50:35.874382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:50:35.874382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:50:35.874382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:50:35.874382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:50:35.874382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:50:35.874382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:50:35.874382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:50:35.874382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:50:35.874382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:50:35.874382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:50:35.874382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:50:35.874382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:50:35.874382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:50:35.875517 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:50:35.875517 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:50:35.875517 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:50:35.875517 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:50:35.875517 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:50:35.875517 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:50:35.875517 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:50:35.875517 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:50:35.875517 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 20:50:35.875517 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:50:35.875517 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 605/1212] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:50:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:50:35.880252 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7950d-2580-45b8-a268-46e16356be19 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:35.977836 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7950d-2580-45b8-a268-46e16356be19 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:36.011747 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4779a5-c257-47a7-b3ab-66119ad989c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:36.012155 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4779a5-c257-47a7-b3ab-66119ad989c6 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:36.012435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4779a5-c257-47a7-b3ab-66119ad989c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:36.012706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4779a5-c257-47a7-b3ab-66119ad989c6 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:36.013423 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4779a5-c257-47a7-b3ab-66119ad989c6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:36.013756 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4779a5-c257-47a7-b3ab-66119ad989c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:36.015194 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4779a5-c257-47a7-b3ab-66119ad989c6 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:36.031932 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 608/1213] 10.222.0.48 () {56 vars in 1143 bytes} [Fri Sep 9 20:50:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:50:36.038202 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3f01f-5166-43b4-9fa7-a3d574cac664 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:36.136756 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3f01f-5166-43b4-9fa7-a3d574cac664 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:36.171165 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb61a0-de6a-4544-af7b-df33295b0c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:36.171769 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb61a0-de6a-4544-af7b-df33295b0c64 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:36.172042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb61a0-de6a-4544-af7b-df33295b0c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:36.172351 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb61a0-de6a-4544-af7b-df33295b0c64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:36.201962 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:36.238268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb61a0-de6a-4544-af7b-df33295b0c64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:36.239704 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb61a0-de6a-4544-af7b-df33295b0c64 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:36.340544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 606/1214] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:50:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:50:36.346466 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4779a5-c257-47a7-b3ab-66119ad989c6 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:36.366115 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:36.444587 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4779a5-c257-47a7-b3ab-66119ad989c6 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:36.478290 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa949336-2dcd-4958-988e-a32b04f81128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:36.478911 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa949336-2dcd-4958-988e-a32b04f81128 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:36.479189 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa949336-2dcd-4958-988e-a32b04f81128 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:36.479460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa949336-2dcd-4958-988e-a32b04f81128 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:36.489135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa949336-2dcd-4958-988e-a32b04f81128 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:36.492142 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa949336-2dcd-4958-988e-a32b04f81128 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:36.504768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-fa949336-2dcd-4958-988e-a32b04f81128 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:50:36.504768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:50:36.504768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:50:36.504768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:50:36.504768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:50:36.504768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:50:36.504768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:50:36.504768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:50:36.504768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:50:36.504768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:50:36.504768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:50:36.504768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:50:36.504768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:50:36.504768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:50:36.504768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:50:36.504768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:50:36.510303 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:50:36.510303 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:50:36.510303 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:50:36.510303 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:50:36.510303 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:50:36.510303 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:50:36.510303 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:50:36.510303 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:50:36.510303 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:50:36.510303 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:50:36.510303 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:50:36.510303 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:50:36.510303 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:50:36.510303 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:50:36.510303 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:50:36.510303 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:50:36.510303 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:50:36.511435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:50:36.511435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:50:36.511435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:50:36.511435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 609/1215] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:50:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:50:36.511897 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb61a0-de6a-4544-af7b-df33295b0c64 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:36.573063 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:36.610467 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb61a0-de6a-4544-af7b-df33295b0c64 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:36.643413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a27d6d9-fc80-48cc-b624-616e1188c5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:36.644652 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27d6d9-fc80-48cc-b624-616e1188c5ee None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:36.644928 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27d6d9-fc80-48cc-b624-616e1188c5ee None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:36.645193 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27d6d9-fc80-48cc-b624-616e1188c5ee None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:36.645825 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a27d6d9-fc80-48cc-b624-616e1188c5ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:36.646139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a27d6d9-fc80-48cc-b624-616e1188c5ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:36.647475 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a27d6d9-fc80-48cc-b624-616e1188c5ee None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:36.658244 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 607/1216] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:50:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:50:36.663939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa949336-2dcd-4958-988e-a32b04f81128 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:36.737918 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:36.767857 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa949336-2dcd-4958-988e-a32b04f81128 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:36.799529 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da5fe4-dd7b-4849-9e3d-9623b3c12a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:36.800262 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da5fe4-dd7b-4849-9e3d-9623b3c12a6a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:36.800543 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da5fe4-dd7b-4849-9e3d-9623b3c12a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:36.800811 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da5fe4-dd7b-4849-9e3d-9623b3c12a6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:36.807423 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da5fe4-dd7b-4849-9e3d-9623b3c12a6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:36.808935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da5fe4-dd7b-4849-9e3d-9623b3c12a6a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:36.821052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-a0da5fe4-dd7b-4849-9e3d-9623b3c12a6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:50:36.821052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:50:36.821052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:50:36.821052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:50:36.821052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:50:36.821052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:50:36.821052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:50:36.821052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:50:36.821052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:50:36.821052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:50:36.821052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:50:36.821052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:50:36.821052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:50:36.821052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:50:36.821052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:50:36.825736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:50:36.825736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:50:36.825736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:50:36.825736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:50:36.825736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:50:36.825736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:50:36.825736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:50:36.825736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:50:36.825736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:50:36.825736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:50:36.825736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:50:36.825736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:50:36.825736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:50:36.825736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:50:36.825736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:50:36.825736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:50:36.825736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:50:36.826884 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:50:36.826884 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:50:36.826884 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:50:36.826884 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:50:36.826884 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:50:36.826884 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:50:36.826884 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:50:36.826884 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:50:36.826884 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 610/1217] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:50:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:50:36.827995 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a27d6d9-fc80-48cc-b624-616e1188c5ee None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:36.926415 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a27d6d9-fc80-48cc-b624-616e1188c5ee None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:36.948530 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:36.960289 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755d61c9-3d9c-48db-9b07-a24b2632a74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:36.960949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d61c9-3d9c-48db-9b07-a24b2632a74a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:36.961229 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d61c9-3d9c-48db-9b07-a24b2632a74a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:36.961497 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d61c9-3d9c-48db-9b07-a24b2632a74a None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:36.962238 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755d61c9-3d9c-48db-9b07-a24b2632a74a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:36.962494 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755d61c9-3d9c-48db-9b07-a24b2632a74a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:36.963921 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755d61c9-3d9c-48db-9b07-a24b2632a74a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:36.975632 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 608/1218] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:50:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:50:36.981765 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da5fe4-dd7b-4849-9e3d-9623b3c12a6a None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:37.082052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da5fe4-dd7b-4849-9e3d-9623b3c12a6a None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:37.109210 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:37.115563 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28be4e4-cab7-414f-9d6c-3612ef17e01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:37.116185 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28be4e4-cab7-414f-9d6c-3612ef17e01e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:37.116465 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28be4e4-cab7-414f-9d6c-3612ef17e01e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:37.116732 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28be4e4-cab7-414f-9d6c-3612ef17e01e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:37.117452 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28be4e4-cab7-414f-9d6c-3612ef17e01e None admin] RBAC: Adding query filter params (role.id=1e000597af164d54a40df2a39c729820, scope.project.id=91319aa782334fb9930770029b74e84a, user.id=2a4f6de2065d4fbea3fa141da760f708) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:37.117757 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28be4e4-cab7-414f-9d6c-3612ef17e01e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:37.119168 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28be4e4-cab7-414f-9d6c-3612ef17e01e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:37.124391 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 611/1219] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:50:36 2022] GET /identity/v3/role_assignments?user.id=2a4f6de2065d4fbea3fa141da760f708&scope.project.id=91319aa782334fb9930770029b74e84a&role.id=1e000597af164d54a40df2a39c729820 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:50:39.225697 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbab1be-6a7f-4d89-9b7f-049628f09106 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:39.226283 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbab1be-6a7f-4d89-9b7f-049628f09106 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:39.226283 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbab1be-6a7f-4d89-9b7f-049628f09106 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:39.227588 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 609/1220] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:50:39.232839 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf037c4-da82-4bb4-8714-85c577284ce3 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:39.246726 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf037c4-da82-4bb4-8714-85c577284ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:39.246726 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf037c4-da82-4bb4-8714-85c577284ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:39.251080 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-2bf037c4-da82-4bb4-8714-85c577284ce3 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:50:39.252305 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2bf037c4-da82-4bb4-8714-85c577284ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:50:39.318956 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 612/1221] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:50:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:50:39.589023 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbab1be-6a7f-4d89-9b7f-049628f09106 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:39.688432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbab1be-6a7f-4d89-9b7f-049628f09106 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:39.722362 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30765f9-fde2-4a5f-9065-896aeaf6f888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:39.722983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30765f9-fde2-4a5f-9065-896aeaf6f888 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:39.723257 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30765f9-fde2-4a5f-9065-896aeaf6f888 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:39.723527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30765f9-fde2-4a5f-9065-896aeaf6f888 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:39.733486 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30765f9-fde2-4a5f-9065-896aeaf6f888 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:50:39.738337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-f30765f9-fde2-4a5f-9065-896aeaf6f888 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:50:39.738337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:50:39.738337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:50:39.738337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:50:39.738337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:50:39.738337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:50:39.738337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:50:39.738337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:50:39.738337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:50:39.738337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:50:39.738337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:50:39.738337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:50:39.738337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:50:39.738337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:50:39.738337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:50:39.738337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:50:39.739408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:50:39.739408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:50:39.739408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:50:39.739408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:50:39.739408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:50:39.739408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:50:39.739408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:50:39.739408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:50:39.739408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:50:39.739408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:50:39.739408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:50:39.739408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:50:39.739408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:50:39.739408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:50:39.739408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:50:39.739408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:50:39.740481 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:50:39.740481 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:50:39.740481 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:50:39.740481 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:50:39.740481 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:50:39.740481 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:50:39.740481 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:50:39.740481 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:50:39.740481 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:50:39.740481 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:50:39.740481 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:50:39.740481 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:50:39.744236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 610/1222] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Sep 9 20:50:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:50:39.745483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf037c4-da82-4bb4-8714-85c577284ce3 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:39.840548 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf037c4-da82-4bb4-8714-85c577284ce3 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:39.862733 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:39.873049 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a75bb5-9dd3-405f-9f6c-64b93b5e300c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:39.873650 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a75bb5-9dd3-405f-9f6c-64b93b5e300c None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:39.873927 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a75bb5-9dd3-405f-9f6c-64b93b5e300c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:39.874200 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a75bb5-9dd3-405f-9f6c-64b93b5e300c None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:39.874857 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a75bb5-9dd3-405f-9f6c-64b93b5e300c None admin] RBAC: Adding query filter params (name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:39.875159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a75bb5-9dd3-405f-9f6c-64b93b5e300c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:39.876553 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a75bb5-9dd3-405f-9f6c-64b93b5e300c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:39.884595 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 613/1223] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Sep 9 20:50:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:50:39.890366 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30765f9-fde2-4a5f-9065-896aeaf6f888 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:39.932439 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:39.988390 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30765f9-fde2-4a5f-9065-896aeaf6f888 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:40.022408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f57fe2-67b9-4dce-a85e-02df3a55ba46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:40.023029 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f57fe2-67b9-4dce-a85e-02df3a55ba46 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:40.023306 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f57fe2-67b9-4dce-a85e-02df3a55ba46 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:40.023571 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f57fe2-67b9-4dce-a85e-02df3a55ba46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:40.033321 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f57fe2-67b9-4dce-a85e-02df3a55ba46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:40.035697 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f57fe2-67b9-4dce-a85e-02df3a55ba46 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:40.048867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-69f57fe2-67b9-4dce-a85e-02df3a55ba46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:50:40.048867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:50:40.048867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:50:40.048867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:50:40.048867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:50:40.048867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:50:40.048867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:50:40.048867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:50:40.048867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:50:40.048867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:50:40.048867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:50:40.048867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:50:40.048867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:50:40.048867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:50:40.048867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:50:40.048867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:50:40.050458 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:50:40.050458 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:50:40.050458 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:50:40.050458 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:50:40.050458 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:50:40.050458 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:50:40.050458 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:50:40.050458 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:50:40.050458 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:50:40.050458 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:50:40.050458 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:50:40.050458 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:50:40.050458 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:50:40.050458 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:50:40.050458 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:50:40.050458 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:50:40.050458 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:50:40.051709 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:50:40.051709 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:50:40.051709 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:50:40.051709 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 611/1224] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:50:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:50:40.056381 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a75bb5-9dd3-405f-9f6c-64b93b5e300c None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:40.154778 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a75bb5-9dd3-405f-9f6c-64b93b5e300c None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:40.187431 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd50ead-9ebf-48c7-bcaa-009505638376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:40.188026 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd50ead-9ebf-48c7-bcaa-009505638376 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:40.188333 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd50ead-9ebf-48c7-bcaa-009505638376 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:40.188605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd50ead-9ebf-48c7-bcaa-009505638376 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:40.189239 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd50ead-9ebf-48c7-bcaa-009505638376 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:40.189794 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd50ead-9ebf-48c7-bcaa-009505638376 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:40.191739 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd50ead-9ebf-48c7-bcaa-009505638376 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:40.202994 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 614/1225] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:50:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:50:40.208732 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f57fe2-67b9-4dce-a85e-02df3a55ba46 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:40.300559 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:40.306954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f57fe2-67b9-4dce-a85e-02df3a55ba46 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:40.339813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133cafd7-1881-4225-bbf6-0b1aa69fbd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:40.340438 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133cafd7-1881-4225-bbf6-0b1aa69fbd9d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:40.340716 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133cafd7-1881-4225-bbf6-0b1aa69fbd9d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:40.340985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133cafd7-1881-4225-bbf6-0b1aa69fbd9d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:40.348532 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133cafd7-1881-4225-bbf6-0b1aa69fbd9d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:40.349958 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133cafd7-1881-4225-bbf6-0b1aa69fbd9d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:40.360623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-133cafd7-1881-4225-bbf6-0b1aa69fbd9d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 20:50:40.360623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:50:40.360623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:50:40.360623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:50:40.360623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:50:40.360623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:50:40.360623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:50:40.360623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:50:40.360623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:50:40.360623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:50:40.360623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:50:40.360623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:50:40.360623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:50:40.360623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:50:40.360623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:50:40.360623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:50:40.361774 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:50:40.361774 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:50:40.361774 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:50:40.361774 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:50:40.361774 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:50:40.361774 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:50:40.361774 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:50:40.361774 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:50:40.361774 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:50:40.361774 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:50:40.361774 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:50:40.361774 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:50:40.361774 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:50:40.361774 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:50:40.361774 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:50:40.361774 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:50:40.361774 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:50:40.361774 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:50:40.363772 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:50:40.363772 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:50:40.363772 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:50:40.363772 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:50:40.363772 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:50:40.363772 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:50:40.363772 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:50:40.363772 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:50:40.363772 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 20:50:40.363772 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:50:40.363772 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 612/1226] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:50:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:50:40.374593 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd50ead-9ebf-48c7-bcaa-009505638376 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:40.413972 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:40.463880 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd50ead-9ebf-48c7-bcaa-009505638376 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:40.496996 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6fa10-52a2-447b-b30c-b81defd29871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:40.497591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6fa10-52a2-447b-b30c-b81defd29871 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:40.497868 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6fa10-52a2-447b-b30c-b81defd29871 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:40.498134 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6fa10-52a2-447b-b30c-b81defd29871 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:40.498845 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d6fa10-52a2-447b-b30c-b81defd29871 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:40.499149 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d6fa10-52a2-447b-b30c-b81defd29871 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:40.500603 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d6fa10-52a2-447b-b30c-b81defd29871 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:40.516780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 615/1227] 10.222.0.48 () {56 vars in 1143 bytes} [Fri Sep 9 20:50:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:50:40.522602 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133cafd7-1881-4225-bbf6-0b1aa69fbd9d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:40.622524 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133cafd7-1881-4225-bbf6-0b1aa69fbd9d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:40.655649 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021fe7f7-995a-4309-ae5f-78ca5a9ffeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:40.656300 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021fe7f7-995a-4309-ae5f-78ca5a9ffeb8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:40.662926 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021fe7f7-995a-4309-ae5f-78ca5a9ffeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:40.662926 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021fe7f7-995a-4309-ae5f-78ca5a9ffeb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:40.668347 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021fe7f7-995a-4309-ae5f-78ca5a9ffeb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:40.669729 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021fe7f7-995a-4309-ae5f-78ca5a9ffeb8 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:40.684485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-021fe7f7-995a-4309-ae5f-78ca5a9ffeb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:50:40.684485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:50:40.684485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:50:40.684485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:50:40.684485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:50:40.684485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:50:40.684485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:50:40.684485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:50:40.684485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:50:40.684485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:50:40.684485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:50:40.684485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:50:40.684485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:50:40.684485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:50:40.684485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:50:40.684485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:50:40.685590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:50:40.685590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:50:40.685590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:50:40.685590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:50:40.685590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:50:40.685590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:50:40.685590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:50:40.685590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:50:40.685590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:50:40.685590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:50:40.685590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:50:40.685590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:50:40.685590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:50:40.685590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:50:40.685590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:50:40.685590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:50:40.685590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:50:40.686534 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:50:40.686534 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:50:40.686534 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:50:40.686534 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 613/1228] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:50:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:50:40.691584 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6fa10-52a2-447b-b30c-b81defd29871 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:40.793248 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6fa10-52a2-447b-b30c-b81defd29871 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:40.795508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:40.826738 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4039fc1-0c30-4cc8-94f1-5bb94e260a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:40.827344 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4039fc1-0c30-4cc8-94f1-5bb94e260a8b None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:40.827634 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4039fc1-0c30-4cc8-94f1-5bb94e260a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:40.827900 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4039fc1-0c30-4cc8-94f1-5bb94e260a8b None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:40.828574 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4039fc1-0c30-4cc8-94f1-5bb94e260a8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:40.828878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4039fc1-0c30-4cc8-94f1-5bb94e260a8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:40.830243 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4039fc1-0c30-4cc8-94f1-5bb94e260a8b None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:40.841575 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 616/1229] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:50:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:50:40.848135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021fe7f7-995a-4309-ae5f-78ca5a9ffeb8 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:40.857408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:40.944716 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021fe7f7-995a-4309-ae5f-78ca5a9ffeb8 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:40.980119 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966dd7e-bc08-4f2c-a0a1-269e41605904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:40.980748 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966dd7e-bc08-4f2c-a0a1-269e41605904 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:40.981940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966dd7e-bc08-4f2c-a0a1-269e41605904 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:40.981940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966dd7e-bc08-4f2c-a0a1-269e41605904 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:40.988150 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f966dd7e-bc08-4f2c-a0a1-269e41605904 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:40.989582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f966dd7e-bc08-4f2c-a0a1-269e41605904 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:41.000559 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-f966dd7e-bc08-4f2c-a0a1-269e41605904 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:50:41.000559 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:50:41.000559 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:50:41.000559 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:50:41.000559 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:50:41.000559 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:50:41.000559 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:50:41.000559 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:50:41.000559 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:50:41.000559 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:50:41.000559 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:50:41.000559 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:50:41.000559 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:50:41.000559 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:50:41.000559 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:50:41.005896 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:50:41.005896 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:50:41.005896 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:50:41.005896 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:50:41.005896 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:50:41.005896 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:50:41.005896 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:50:41.005896 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:50:41.005896 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:50:41.005896 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:50:41.005896 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:50:41.005896 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:50:41.005896 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:50:41.005896 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:50:41.005896 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:50:41.005896 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:50:41.005896 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:50:41.006979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:50:41.006979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:50:41.006979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:50:41.006979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:50:41.006979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:50:41.006979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:50:41.006979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:50:41.006979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:50:41.006979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 614/1230] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:50:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:50:41.008922 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4039fc1-0c30-4cc8-94f1-5bb94e260a8b None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:41.104688 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4039fc1-0c30-4cc8-94f1-5bb94e260a8b None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:41.136605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607df1f3-8bfd-48e7-9483-585f82c2cb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:41.137181 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607df1f3-8bfd-48e7-9483-585f82c2cb30 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:41.137452 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607df1f3-8bfd-48e7-9483-585f82c2cb30 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:41.137716 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607df1f3-8bfd-48e7-9483-585f82c2cb30 None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:41.138396 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607df1f3-8bfd-48e7-9483-585f82c2cb30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:41.138698 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607df1f3-8bfd-48e7-9483-585f82c2cb30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:41.140121 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607df1f3-8bfd-48e7-9483-585f82c2cb30 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:41.151899 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 617/1231] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:50:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:50:41.157841 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966dd7e-bc08-4f2c-a0a1-269e41605904 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:41.221318 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:41.254842 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966dd7e-bc08-4f2c-a0a1-269e41605904 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:41.287027 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552ae15-058a-485b-97e8-2dd3e01c02ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:41.287672 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552ae15-058a-485b-97e8-2dd3e01c02ca None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:41.287950 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552ae15-058a-485b-97e8-2dd3e01c02ca None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:41.288326 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552ae15-058a-485b-97e8-2dd3e01c02ca None admin] PATH_INFO: `/v3/projects/91319aa782334fb9930770029b74e84a/users/2a4f6de2065d4fbea3fa141da760f708/roles/1e000597af164d54a40df2a39c729820` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:41.302264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5552ae15-058a-485b-97e8-2dd3e01c02ca None admin] RBAC: Authorizing `identity:create_grant(project_id=91319aa782334fb9930770029b74e84a, user_id=2a4f6de2065d4fbea3fa141da760f708, role_id=1e000597af164d54a40df2a39c729820)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:41.303705 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5552ae15-058a-485b-97e8-2dd3e01c02ca None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:41.391896 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 615/1232] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Sep 9 20:50:41 2022] PUT /identity/v3/projects/91319aa782334fb9930770029b74e84a/users/2a4f6de2065d4fbea3fa141da760f708/roles/1e000597af164d54a40df2a39c729820 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:50:43.469962 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdafb8d-0757-40e9-b1f1-6d1fd02a9a67 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:43.470529 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdafb8d-0757-40e9-b1f1-6d1fd02a9a67 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:43.470529 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdafb8d-0757-40e9-b1f1-6d1fd02a9a67 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:43.472494 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 618/1233] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:50:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:50:43.477636 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26655ed2-fc7e-40c1-9208-25a42ddf7da1 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:43.482750 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26655ed2-fc7e-40c1-9208-25a42ddf7da1 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:43.482750 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26655ed2-fc7e-40c1-9208-25a42ddf7da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:43.497228 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-26655ed2-fc7e-40c1-9208-25a42ddf7da1 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:50:43.498434 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-26655ed2-fc7e-40c1-9208-25a42ddf7da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:50:43.565389 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 616/1234] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:50:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:50:43.832321 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdafb8d-0757-40e9-b1f1-6d1fd02a9a67 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:43.857904 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:43.931208 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdafb8d-0757-40e9-b1f1-6d1fd02a9a67 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:43.965309 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb27fb1-6358-476d-8db8-cf8246dc8db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:43.965937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb27fb1-6358-476d-8db8-cf8246dc8db3 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:43.966214 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb27fb1-6358-476d-8db8-cf8246dc8db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:43.966484 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb27fb1-6358-476d-8db8-cf8246dc8db3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:43.977011 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb27fb1-6358-476d-8db8-cf8246dc8db3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:50:43.981817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-5eb27fb1-6358-476d-8db8-cf8246dc8db3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:50:43.981817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:50:43.981817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:50:43.981817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:50:43.981817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:50:43.981817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:50:43.981817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:50:43.981817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:50:43.981817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:50:43.981817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:50:43.981817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:50:43.981817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:50:43.981817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:50:43.981817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:50:43.981817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:50:43.981817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:50:43.982770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:50:43.982770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:50:43.982770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:50:43.982770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:50:43.982770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:50:43.982770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:50:43.982770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:50:43.982770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:50:43.982770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:50:43.982770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:50:43.982770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:50:43.982770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:50:43.982770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:50:43.982770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:50:43.982770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:50:43.982770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:50:43.983924 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:50:43.983924 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:50:43.983924 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:50:43.983924 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:50:43.983924 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:50:43.983924 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:50:43.983924 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:50:43.983924 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:50:43.983924 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:50:43.983924 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:50:43.983924 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 20:50:43.983924 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:50:43.983924 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 619/1235] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Sep 9 20:50:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:50:43.988937 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26655ed2-fc7e-40c1-9208-25a42ddf7da1 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:44.036678 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:44.087336 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26655ed2-fc7e-40c1-9208-25a42ddf7da1 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:44.120757 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee1d6f-9085-4606-acbb-03e26b3d0930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:44.121354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee1d6f-9085-4606-acbb-03e26b3d0930 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:44.121633 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee1d6f-9085-4606-acbb-03e26b3d0930 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:44.121916 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee1d6f-9085-4606-acbb-03e26b3d0930 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:44.122730 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee1d6f-9085-4606-acbb-03e26b3d0930 None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:44.123035 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee1d6f-9085-4606-acbb-03e26b3d0930 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:44.124431 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee1d6f-9085-4606-acbb-03e26b3d0930 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:44.132584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 617/1236] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Sep 9 20:50:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:50:44.138497 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb27fb1-6358-476d-8db8-cf8246dc8db3 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:44.222254 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:44.237535 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb27fb1-6358-476d-8db8-cf8246dc8db3 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:44.270837 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832e612-eea5-474d-8fa9-b70cd6e78592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:44.271472 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832e612-eea5-474d-8fa9-b70cd6e78592 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:44.271745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832e612-eea5-474d-8fa9-b70cd6e78592 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:44.272014 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832e612-eea5-474d-8fa9-b70cd6e78592 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:44.281600 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2832e612-eea5-474d-8fa9-b70cd6e78592 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:44.283102 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2832e612-eea5-474d-8fa9-b70cd6e78592 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:44.297190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-2832e612-eea5-474d-8fa9-b70cd6e78592 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:50:44.297190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:50:44.297190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:50:44.297190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:50:44.297190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:50:44.297190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:50:44.297190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:50:44.297190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:50:44.297190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:50:44.297190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:50:44.297190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:50:44.297190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:50:44.297190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:50:44.297190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:50:44.297190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:50:44.297190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:50:44.298305 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:50:44.298305 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:50:44.298305 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:50:44.298305 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:50:44.298305 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:50:44.298305 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:50:44.298305 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:50:44.298305 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:50:44.298305 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:50:44.298305 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:50:44.298305 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:50:44.298305 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:50:44.298305 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:50:44.298305 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:50:44.298305 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:50:44.298305 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:50:44.298305 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:50:44.299426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:50:44.299426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:50:44.299426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:50:44.299426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 620/1237] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:50:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:50:44.304284 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee1d6f-9085-4606-acbb-03e26b3d0930 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:44.401217 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee1d6f-9085-4606-acbb-03e26b3d0930 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:44.409749 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:44.433003 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec7af9a-f8d7-44c7-80b3-f373fd19947c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:44.433598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7af9a-f8d7-44c7-80b3-f373fd19947c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:44.433874 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7af9a-f8d7-44c7-80b3-f373fd19947c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:44.434141 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7af9a-f8d7-44c7-80b3-f373fd19947c None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:44.434777 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec7af9a-f8d7-44c7-80b3-f373fd19947c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:44.435073 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec7af9a-f8d7-44c7-80b3-f373fd19947c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:44.436455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec7af9a-f8d7-44c7-80b3-f373fd19947c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:44.448577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 618/1238] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:50:44.454283 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832e612-eea5-474d-8fa9-b70cd6e78592 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:44.567006 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832e612-eea5-474d-8fa9-b70cd6e78592 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:44.599584 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153266f2-7c7a-4044-8b13-441281c0a752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:44.600237 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153266f2-7c7a-4044-8b13-441281c0a752 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:44.600513 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153266f2-7c7a-4044-8b13-441281c0a752 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:44.600778 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153266f2-7c7a-4044-8b13-441281c0a752 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:44.603358 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:44.608160 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153266f2-7c7a-4044-8b13-441281c0a752 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:44.609584 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153266f2-7c7a-4044-8b13-441281c0a752 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:44.621354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-153266f2-7c7a-4044-8b13-441281c0a752 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 20:50:44.621354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:50:44.621354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:50:44.621354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:50:44.621354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:50:44.621354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:50:44.621354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:50:44.621354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:50:44.621354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:50:44.621354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:50:44.621354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:50:44.621354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:50:44.621354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:50:44.621354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:50:44.621354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:50:44.621354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:50:44.622501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:50:44.622501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:50:44.622501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:50:44.622501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:50:44.622501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:50:44.622501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:50:44.622501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:50:44.622501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:50:44.622501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:50:44.622501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:50:44.622501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:50:44.622501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:50:44.622501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:50:44.622501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:50:44.622501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:50:44.622501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:50:44.622501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:50:44.622501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:50:44.623605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:50:44.623605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:50:44.623605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:50:44.623605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:50:44.623605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:50:44.623605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:50:44.623605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:50:44.623605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:50:44.623605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 20:50:44.623605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:50:44.623605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 621/1239] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:50:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:50:44.628948 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec7af9a-f8d7-44c7-80b3-f373fd19947c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:44.726268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec7af9a-f8d7-44c7-80b3-f373fd19947c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:44.766421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838861db-031a-48ca-9fef-eb9e35a35f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:44.767026 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838861db-031a-48ca-9fef-eb9e35a35f3c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:44.767304 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838861db-031a-48ca-9fef-eb9e35a35f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:44.767573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838861db-031a-48ca-9fef-eb9e35a35f3c None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:44.768339 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838861db-031a-48ca-9fef-eb9e35a35f3c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:44.774138 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838861db-031a-48ca-9fef-eb9e35a35f3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:44.774138 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838861db-031a-48ca-9fef-eb9e35a35f3c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:44.788020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 619/1240] 10.222.0.48 () {56 vars in 1143 bytes} [Fri Sep 9 20:50:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:50:44.793910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153266f2-7c7a-4044-8b13-441281c0a752 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:44.906484 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153266f2-7c7a-4044-8b13-441281c0a752 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:44.939435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c1749-2520-421c-8a82-4e52fa0895d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:44.940339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c1749-2520-421c-8a82-4e52fa0895d1 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:44.940617 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c1749-2520-421c-8a82-4e52fa0895d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:44.940886 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c1749-2520-421c-8a82-4e52fa0895d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:45.001011 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:45.007731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c1749-2520-421c-8a82-4e52fa0895d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:45.009217 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c1749-2520-421c-8a82-4e52fa0895d1 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:45.103993 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 622/1241] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:50:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:50:45.109842 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838861db-031a-48ca-9fef-eb9e35a35f3c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:45.136477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:45.211447 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838861db-031a-48ca-9fef-eb9e35a35f3c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:45.244603 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cad15b-60a1-48b8-8dcb-c6d1de95d276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:45.245229 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cad15b-60a1-48b8-8dcb-c6d1de95d276 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:45.245535 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cad15b-60a1-48b8-8dcb-c6d1de95d276 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:45.245808 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cad15b-60a1-48b8-8dcb-c6d1de95d276 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:45.255719 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cad15b-60a1-48b8-8dcb-c6d1de95d276 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:45.257122 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cad15b-60a1-48b8-8dcb-c6d1de95d276 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:45.271817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-70cad15b-60a1-48b8-8dcb-c6d1de95d276 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:50:45.271817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:50:45.271817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:50:45.271817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:50:45.271817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:50:45.271817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:50:45.271817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:50:45.271817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:50:45.271817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:50:45.271817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:50:45.271817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:50:45.271817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:50:45.271817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:50:45.271817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:50:45.271817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:50:45.271817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:50:45.272980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:50:45.272980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:50:45.272980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:50:45.272980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:50:45.272980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:50:45.272980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:50:45.272980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:50:45.272980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:50:45.272980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:50:45.272980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:50:45.272980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:50:45.272980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:50:45.272980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:50:45.272980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:50:45.272980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:50:45.272980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:50:45.272980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:50:45.274293 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:50:45.274293 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:50:45.274293 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:50:45.274293 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 620/1242] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:50:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:50:45.278664 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c1749-2520-421c-8a82-4e52fa0895d1 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:45.372902 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:45.379435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c1749-2520-421c-8a82-4e52fa0895d1 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:45.414545 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227ec59-dd66-4352-902e-e9ee60c9f64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:45.415152 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227ec59-dd66-4352-902e-e9ee60c9f64a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:45.415426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227ec59-dd66-4352-902e-e9ee60c9f64a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:45.415696 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227ec59-dd66-4352-902e-e9ee60c9f64a None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:45.416365 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d227ec59-dd66-4352-902e-e9ee60c9f64a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:45.416664 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d227ec59-dd66-4352-902e-e9ee60c9f64a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:45.418013 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d227ec59-dd66-4352-902e-e9ee60c9f64a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:45.429366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 623/1243] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:50:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:50:45.435163 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cad15b-60a1-48b8-8dcb-c6d1de95d276 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:45.515972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:45.542683 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cad15b-60a1-48b8-8dcb-c6d1de95d276 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:45.577508 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b96bd-4959-4029-bcbc-07853cc406a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:45.578140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b96bd-4959-4029-bcbc-07853cc406a4 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:45.578418 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b96bd-4959-4029-bcbc-07853cc406a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:45.578689 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b96bd-4959-4029-bcbc-07853cc406a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:45.586080 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b96bd-4959-4029-bcbc-07853cc406a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:45.587598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b96bd-4959-4029-bcbc-07853cc406a4 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:45.600855 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-1e3b96bd-4959-4029-bcbc-07853cc406a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:50:45.600855 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:50:45.600855 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:50:45.600855 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:50:45.600855 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:50:45.600855 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:50:45.600855 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:50:45.600855 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:50:45.600855 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:50:45.600855 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:50:45.600855 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:50:45.600855 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:50:45.600855 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:50:45.600855 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:50:45.600855 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:50:45.609106 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:50:45.609106 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:50:45.609106 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:50:45.609106 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:50:45.609106 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:50:45.609106 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:50:45.609106 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:50:45.609106 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:50:45.609106 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:50:45.609106 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:50:45.609106 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:50:45.609106 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:50:45.609106 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:50:45.609106 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:50:45.609106 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:50:45.609106 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:50:45.609106 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:50:45.610210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:50:45.610210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:50:45.610210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:50:45.610210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:50:45.610210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:50:45.610210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:50:45.610210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:50:45.610210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:50:45.610210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 621/1244] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:50:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:50:45.610831 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227ec59-dd66-4352-902e-e9ee60c9f64a None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:45.705542 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227ec59-dd66-4352-902e-e9ee60c9f64a None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:45.738334 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f9c3b-940e-4d36-a7ee-53551aa1d07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:45.738931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f9c3b-940e-4d36-a7ee-53551aa1d07f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:45.739216 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f9c3b-940e-4d36-a7ee-53551aa1d07f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:45.739485 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f9c3b-940e-4d36-a7ee-53551aa1d07f None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:45.740203 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f9c3b-940e-4d36-a7ee-53551aa1d07f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:45.740515 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f9c3b-940e-4d36-a7ee-53551aa1d07f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:45.741930 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f9c3b-940e-4d36-a7ee-53551aa1d07f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:45.753960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 624/1245] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:50:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:50:45.760219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b96bd-4959-4029-bcbc-07853cc406a4 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:45.859015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b96bd-4959-4029-bcbc-07853cc406a4 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:45.891516 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63440f10-06ce-45ce-904c-3ca9ce8e3700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:45.892131 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63440f10-06ce-45ce-904c-3ca9ce8e3700 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:45.892411 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63440f10-06ce-45ce-904c-3ca9ce8e3700 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:45.892677 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63440f10-06ce-45ce-904c-3ca9ce8e3700 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:45.894015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63440f10-06ce-45ce-904c-3ca9ce8e3700 None admin] RBAC: Adding query filter params (role.id=1e000597af164d54a40df2a39c729820, scope.project.id=91319aa782334fb9930770029b74e84a, user.id=2a4f6de2065d4fbea3fa141da760f708) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:45.894315 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63440f10-06ce-45ce-904c-3ca9ce8e3700 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:45.895774 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63440f10-06ce-45ce-904c-3ca9ce8e3700 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:45.901373 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 622/1246] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:50:45 2022] GET /identity/v3/role_assignments?user.id=2a4f6de2065d4fbea3fa141da760f708&scope.project.id=91319aa782334fb9930770029b74e84a&role.id=1e000597af164d54a40df2a39c729820 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:50:48.135215 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc3a22-ead3-43ce-82e1-3d7c4e1f24bc None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:48.135753 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc3a22-ead3-43ce-82e1-3d7c4e1f24bc None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:48.135753 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc3a22-ead3-43ce-82e1-3d7c4e1f24bc None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:48.137111 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 625/1247] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:50:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:50:48.142571 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe778c-e0d5-4ff4-89ac-ec478b434879 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:48.142979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe778c-e0d5-4ff4-89ac-ec478b434879 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:48.143113 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe778c-e0d5-4ff4-89ac-ec478b434879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:48.149470 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:48.162071 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9cfe778c-e0d5-4ff4-89ac-ec478b434879 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:50:48.163249 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9cfe778c-e0d5-4ff4-89ac-ec478b434879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:50:48.230416 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 623/1248] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:50:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 20:50:48.499383 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc3a22-ead3-43ce-82e1-3d7c4e1f24bc None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:48.517315 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:48.601177 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc3a22-ead3-43ce-82e1-3d7c4e1f24bc None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:48.633694 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1741fc1-d97c-47da-bdf8-f3e94853fcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:48.634400 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1741fc1-d97c-47da-bdf8-f3e94853fcbe None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:48.634774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1741fc1-d97c-47da-bdf8-f3e94853fcbe None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:48.635058 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1741fc1-d97c-47da-bdf8-f3e94853fcbe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:48.646662 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1741fc1-d97c-47da-bdf8-f3e94853fcbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:50:48.651362 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-e1741fc1-d97c-47da-bdf8-f3e94853fcbe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:50:48.651362 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:50:48.651362 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:50:48.651362 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:50:48.651362 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:50:48.651362 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:50:48.651362 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:50:48.651362 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:50:48.651362 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:50:48.651362 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:50:48.651362 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:50:48.651362 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:50:48.651362 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:50:48.651362 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:50:48.651362 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:50:48.651362 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:50:48.652512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:50:48.652512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:50:48.652512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:50:48.652512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:50:48.652512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:50:48.652512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:50:48.652512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:50:48.652512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:50:48.652512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:50:48.652512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:50:48.652512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:50:48.652512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:50:48.652512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:50:48.652512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:50:48.652512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:50:48.652512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:50:48.653553 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:50:48.653553 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:50:48.653553 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:50:48.653553 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:50:48.653553 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:50:48.653553 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:50:48.653553 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:50:48.653553 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:50:48.653553 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:50:48.653553 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:50:48.653553 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:50:48.653553 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:50:48.653553 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 626/1249] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:50:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:50:48.658515 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe778c-e0d5-4ff4-89ac-ec478b434879 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:48.755169 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe778c-e0d5-4ff4-89ac-ec478b434879 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:48.790981 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:48.795758 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e7585-867d-49e2-9c61-f382e403f2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:48.796394 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e7585-867d-49e2-9c61-f382e403f2e1 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:48.796728 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e7585-867d-49e2-9c61-f382e403f2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:48.797003 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e7585-867d-49e2-9c61-f382e403f2e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:48.797662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02e7585-867d-49e2-9c61-f382e403f2e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:48.797964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02e7585-867d-49e2-9c61-f382e403f2e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:48.799334 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02e7585-867d-49e2-9c61-f382e403f2e1 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:48.813406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 624/1250] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:50:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:50:48.820123 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1741fc1-d97c-47da-bdf8-f3e94853fcbe None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:48.896209 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:48.917474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1741fc1-d97c-47da-bdf8-f3e94853fcbe None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:48.950662 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a34e79-9e0b-4ecb-b615-a465257d0a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:48.951326 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a34e79-9e0b-4ecb-b615-a465257d0a7a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:48.951819 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a34e79-9e0b-4ecb-b615-a465257d0a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:48.952287 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a34e79-9e0b-4ecb-b615-a465257d0a7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:48.963629 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a34e79-9e0b-4ecb-b615-a465257d0a7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:48.965097 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a34e79-9e0b-4ecb-b615-a465257d0a7a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:48.979271 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-b8a34e79-9e0b-4ecb-b615-a465257d0a7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:50:48.979271 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:50:48.979271 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:50:48.979271 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:50:48.979271 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:50:48.979271 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:50:48.979271 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:50:48.979271 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:50:48.979271 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:50:48.979271 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:50:48.979271 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:50:48.979271 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:50:48.979271 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:50:48.979271 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:50:48.979271 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:50:48.979271 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:50:48.980373 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:50:48.980373 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:50:48.980373 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:50:48.980373 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:50:48.980373 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:50:48.980373 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:50:48.980373 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:50:48.980373 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:50:48.980373 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:50:48.980373 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:50:48.980373 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:50:48.980373 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:50:48.980373 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:50:48.980373 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:50:48.980373 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:50:48.980373 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:50:48.980373 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:50:48.981264 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:50:48.981264 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:50:48.981264 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:50:48.981264 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 627/1251] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:50:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:50:48.986357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e7585-867d-49e2-9c61-f382e403f2e1 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:49.085541 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e7585-867d-49e2-9c61-f382e403f2e1 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:49.118556 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad239588-b188-4c26-8c96-eb525c8dd6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:49.119155 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad239588-b188-4c26-8c96-eb525c8dd6f0 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:49.119435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad239588-b188-4c26-8c96-eb525c8dd6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:49.119708 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad239588-b188-4c26-8c96-eb525c8dd6f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:49.120414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad239588-b188-4c26-8c96-eb525c8dd6f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:49.120741 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad239588-b188-4c26-8c96-eb525c8dd6f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:49.122160 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad239588-b188-4c26-8c96-eb525c8dd6f0 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:49.134192 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 625/1252] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:50:49.139946 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a34e79-9e0b-4ecb-b615-a465257d0a7a None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:49.236302 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a34e79-9e0b-4ecb-b615-a465257d0a7a None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:49.261657 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:49.269052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5387813-2e19-4260-940f-0112a9622c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:49.269662 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5387813-2e19-4260-940f-0112a9622c8e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:49.269934 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5387813-2e19-4260-940f-0112a9622c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:49.270200 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5387813-2e19-4260-940f-0112a9622c8e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:49.277240 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5387813-2e19-4260-940f-0112a9622c8e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:49.278649 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5387813-2e19-4260-940f-0112a9622c8e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:49.289468 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-e5387813-2e19-4260-940f-0112a9622c8e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 20:50:49.289468 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:50:49.289468 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:50:49.289468 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:50:49.289468 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:50:49.289468 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:50:49.289468 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:50:49.289468 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:50:49.289468 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:50:49.289468 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:50:49.289468 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:50:49.289468 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:50:49.289468 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:50:49.289468 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:50:49.289468 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:50:49.289468 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:50:49.290676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:50:49.290676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:50:49.290676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:50:49.290676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:50:49.290676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:50:49.290676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:50:49.290676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:50:49.290676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:50:49.290676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:50:49.290676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:50:49.290676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:50:49.290676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:50:49.290676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:50:49.290676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:50:49.290676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:50:49.290676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:50:49.290676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:50:49.290676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:50:49.292097 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:50:49.292097 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:50:49.292097 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:50:49.292097 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:50:49.292097 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:50:49.292097 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:50:49.292097 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:50:49.292097 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:50:49.292097 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 20:50:49.292097 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:50:49.292097 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 628/1253] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:50:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:50:49.296449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad239588-b188-4c26-8c96-eb525c8dd6f0 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:49.352852 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:49.396871 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad239588-b188-4c26-8c96-eb525c8dd6f0 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:49.430713 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893cbc61-05f4-48d6-9367-8c8397b6f78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:49.431318 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893cbc61-05f4-48d6-9367-8c8397b6f78f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:49.431599 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893cbc61-05f4-48d6-9367-8c8397b6f78f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:49.431873 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893cbc61-05f4-48d6-9367-8c8397b6f78f None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:49.432631 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893cbc61-05f4-48d6-9367-8c8397b6f78f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:49.432941 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893cbc61-05f4-48d6-9367-8c8397b6f78f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:49.434383 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893cbc61-05f4-48d6-9367-8c8397b6f78f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:49.457975 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 626/1254] 10.222.0.48 () {56 vars in 1143 bytes} [Fri Sep 9 20:50:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:50:49.459176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5387813-2e19-4260-940f-0112a9622c8e None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:49.558741 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5387813-2e19-4260-940f-0112a9622c8e None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:49.595710 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d956c06-9267-43ae-be61-6bb83111f8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:49.596836 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d956c06-9267-43ae-be61-6bb83111f8de None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:49.597146 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d956c06-9267-43ae-be61-6bb83111f8de None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:49.597413 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d956c06-9267-43ae-be61-6bb83111f8de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:49.651473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:49.664098 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d956c06-9267-43ae-be61-6bb83111f8de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:49.665634 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d956c06-9267-43ae-be61-6bb83111f8de None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:49.759511 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 629/1255] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:50:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:50:49.765365 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893cbc61-05f4-48d6-9367-8c8397b6f78f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:49.865718 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893cbc61-05f4-48d6-9367-8c8397b6f78f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:49.878219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:49.899071 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e6016-3e8b-4e18-be6b-b7f6e4ffa047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:49.899695 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e6016-3e8b-4e18-be6b-b7f6e4ffa047 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:49.899976 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e6016-3e8b-4e18-be6b-b7f6e4ffa047 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:49.900275 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e6016-3e8b-4e18-be6b-b7f6e4ffa047 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:49.910521 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e6016-3e8b-4e18-be6b-b7f6e4ffa047 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:49.911973 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e6016-3e8b-4e18-be6b-b7f6e4ffa047 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:49.926834 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-680e6016-3e8b-4e18-be6b-b7f6e4ffa047 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:50:49.926834 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:50:49.926834 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:50:49.926834 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:50:49.926834 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:50:49.926834 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:50:49.926834 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:50:49.926834 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:50:49.926834 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:50:49.926834 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:50:49.926834 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:50:49.926834 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:50:49.926834 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:50:49.926834 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:50:49.926834 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:50:49.926834 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:50:49.927985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:50:49.927985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:50:49.927985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:50:49.927985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:50:49.927985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:50:49.927985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:50:49.927985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:50:49.927985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:50:49.927985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:50:49.927985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:50:49.927985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:50:49.927985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:50:49.927985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:50:49.927985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:50:49.927985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:50:49.927985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:50:49.927985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:50:49.929260 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:50:49.929260 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:50:49.929260 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:50:49.929260 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 627/1256] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:50:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:50:49.933723 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d956c06-9267-43ae-be61-6bb83111f8de None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:50.020620 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:50.031959 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d956c06-9267-43ae-be61-6bb83111f8de None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:50.064196 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08149b8-72d3-4f84-aead-3e733a23415c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:50.065074 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08149b8-72d3-4f84-aead-3e733a23415c None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:50.065353 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08149b8-72d3-4f84-aead-3e733a23415c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:50.065621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08149b8-72d3-4f84-aead-3e733a23415c None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:50.066256 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08149b8-72d3-4f84-aead-3e733a23415c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:50.066552 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08149b8-72d3-4f84-aead-3e733a23415c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:50.067892 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08149b8-72d3-4f84-aead-3e733a23415c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:50.078851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 630/1257] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:50:50.084858 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e6016-3e8b-4e18-be6b-b7f6e4ffa047 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:50.185184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e6016-3e8b-4e18-be6b-b7f6e4ffa047 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:50.219833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70ec57-56f7-40d7-8c01-7cecbd7a8a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:50.220488 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70ec57-56f7-40d7-8c01-7cecbd7a8a11 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:50.220768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70ec57-56f7-40d7-8c01-7cecbd7a8a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:50.221042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70ec57-56f7-40d7-8c01-7cecbd7a8a11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:50.227569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd70ec57-56f7-40d7-8c01-7cecbd7a8a11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:50.229034 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd70ec57-56f7-40d7-8c01-7cecbd7a8a11 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:50.239900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-bd70ec57-56f7-40d7-8c01-7cecbd7a8a11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:50:50.239900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:50:50.239900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:50:50.239900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:50:50.239900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:50:50.239900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:50:50.239900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:50:50.239900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:50:50.239900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:50:50.239900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:50:50.239900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:50:50.239900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:50:50.239900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:50:50.239900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:50:50.239900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:50:50.240825 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:50:50.240825 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:50:50.240825 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:50:50.240825 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:50:50.240825 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:50:50.240825 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:50:50.240825 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:50:50.240825 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:50:50.240825 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:50:50.240825 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:50:50.240825 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:50:50.240825 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:50:50.240825 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:50:50.240825 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:50:50.240825 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:50:50.240825 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:50:50.240825 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:50:50.241808 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:50:50.241808 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:50:50.241808 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:50:50.241808 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:50:50.241808 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:50:50.241808 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:50:50.241808 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:50:50.241808 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:50:50.241808 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 628/1258] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:50:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:50:50.246906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08149b8-72d3-4f84-aead-3e733a23415c None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:50.346851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08149b8-72d3-4f84-aead-3e733a23415c None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:50.380021 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bfa6a6-e0d9-4629-ae65-af2d68698add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:50.380642 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bfa6a6-e0d9-4629-ae65-af2d68698add None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:50.380919 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bfa6a6-e0d9-4629-ae65-af2d68698add None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:50.381187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bfa6a6-e0d9-4629-ae65-af2d68698add None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:50.381869 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bfa6a6-e0d9-4629-ae65-af2d68698add None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:50.382168 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bfa6a6-e0d9-4629-ae65-af2d68698add None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:50.383581 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bfa6a6-e0d9-4629-ae65-af2d68698add None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:50.395718 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 631/1259] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:50:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:50:50.401817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70ec57-56f7-40d7-8c01-7cecbd7a8a11 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:50.427208 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:50.501875 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70ec57-56f7-40d7-8c01-7cecbd7a8a11 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:50.535556 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0de57b-5f70-42e6-933c-d038ea5c7268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:50.536180 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0de57b-5f70-42e6-933c-d038ea5c7268 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:50.536463 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0de57b-5f70-42e6-933c-d038ea5c7268 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:50.536907 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0de57b-5f70-42e6-933c-d038ea5c7268 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:50.537630 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0de57b-5f70-42e6-933c-d038ea5c7268 None admin] RBAC: Adding query filter params (role.id=fcbf519fbc84458f913671a5a1d0914c, scope.project.id=91319aa782334fb9930770029b74e84a, user.id=2a4f6de2065d4fbea3fa141da760f708) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:50.537940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0de57b-5f70-42e6-933c-d038ea5c7268 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:50.539369 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0de57b-5f70-42e6-933c-d038ea5c7268 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:50.544651 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 629/1260] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:50:50 2022] GET /identity/v3/role_assignments?user.id=2a4f6de2065d4fbea3fa141da760f708&scope.project.id=91319aa782334fb9930770029b74e84a&role.id=fcbf519fbc84458f913671a5a1d0914c => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:50:52.666849 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee6813-1bb4-4361-99c8-bc725e91270d None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:52.667424 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee6813-1bb4-4361-99c8-bc725e91270d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:52.667424 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee6813-1bb4-4361-99c8-bc725e91270d None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:52.668863 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 632/1261] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:50:52.674210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6eac21-72a2-49fd-8447-88df329d1424 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:52.675196 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6eac21-72a2-49fd-8447-88df329d1424 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:52.675410 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6eac21-72a2-49fd-8447-88df329d1424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:52.694911 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-8f6eac21-72a2-49fd-8447-88df329d1424 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:50:52.696138 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8f6eac21-72a2-49fd-8447-88df329d1424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:50:52.764583 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 630/1262] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:50:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:50:53.036091 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee6813-1bb4-4361-99c8-bc725e91270d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:53.046959 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:53.136551 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee6813-1bb4-4361-99c8-bc725e91270d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:53.169182 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28110a1a-b69c-4aac-83e3-cdbcfb5733b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:53.169804 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28110a1a-b69c-4aac-83e3-cdbcfb5733b4 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:53.170084 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28110a1a-b69c-4aac-83e3-cdbcfb5733b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:53.170354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28110a1a-b69c-4aac-83e3-cdbcfb5733b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:53.180611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28110a1a-b69c-4aac-83e3-cdbcfb5733b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:50:53.185473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-28110a1a-b69c-4aac-83e3-cdbcfb5733b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:50:53.185473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:50:53.185473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:50:53.185473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:50:53.185473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:50:53.185473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:50:53.185473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:50:53.185473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:50:53.185473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:50:53.185473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:50:53.185473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:50:53.185473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:50:53.185473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:50:53.185473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:50:53.185473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:50:53.185473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:50:53.186570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:50:53.186570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:50:53.186570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:50:53.186570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:50:53.186570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:50:53.186570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:50:53.186570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:50:53.186570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:50:53.186570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:50:53.186570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:50:53.186570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:50:53.186570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:50:53.186570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:50:53.186570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:50:53.186570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:50:53.186570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:50:53.187518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:50:53.187518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:50:53.187518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:50:53.187518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:50:53.187518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:50:53.187518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:50:53.187518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:50:53.187518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:50:53.187518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:50:53.187518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:50:53.187518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:50:53.187518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:50:53.187518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 633/1263] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:50:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:50:53.193029 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6eac21-72a2-49fd-8447-88df329d1424 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:53.210635 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:53.289178 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6eac21-72a2-49fd-8447-88df329d1424 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:53.323320 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44caa1e-a782-4ab2-923d-fa4ee8ec686e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:53.323888 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44caa1e-a782-4ab2-923d-fa4ee8ec686e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:53.324195 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44caa1e-a782-4ab2-923d-fa4ee8ec686e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:53.324469 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44caa1e-a782-4ab2-923d-fa4ee8ec686e None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:53.325126 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44caa1e-a782-4ab2-923d-fa4ee8ec686e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:53.325429 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44caa1e-a782-4ab2-923d-fa4ee8ec686e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:53.326791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44caa1e-a782-4ab2-923d-fa4ee8ec686e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:53.334809 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 631/1264] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:50:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:50:53.340797 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28110a1a-b69c-4aac-83e3-cdbcfb5733b4 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:53.406373 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:53.439194 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28110a1a-b69c-4aac-83e3-cdbcfb5733b4 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:53.473193 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e5f36-21d1-4119-8528-490ad74dd252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:53.473822 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e5f36-21d1-4119-8528-490ad74dd252 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:53.474101 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e5f36-21d1-4119-8528-490ad74dd252 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:53.474371 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e5f36-21d1-4119-8528-490ad74dd252 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:53.484516 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406e5f36-21d1-4119-8528-490ad74dd252 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:53.485953 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406e5f36-21d1-4119-8528-490ad74dd252 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:53.501041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-406e5f36-21d1-4119-8528-490ad74dd252 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:50:53.501041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:50:53.501041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:50:53.501041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:50:53.501041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:50:53.501041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:50:53.501041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:50:53.501041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:50:53.501041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:50:53.501041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:50:53.501041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:50:53.501041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:50:53.501041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:50:53.501041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:50:53.501041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:50:53.501041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:50:53.502159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:50:53.502159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:50:53.502159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:50:53.502159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:50:53.502159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:50:53.502159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:50:53.502159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:50:53.502159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:50:53.502159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:50:53.502159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:50:53.502159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:50:53.502159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:50:53.502159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:50:53.502159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:50:53.502159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:50:53.502159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:50:53.502159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:50:53.503404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:50:53.503404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:50:53.503404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:50:53.503404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 634/1265] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:50:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:50:53.507919 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44caa1e-a782-4ab2-923d-fa4ee8ec686e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:53.588653 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:53.607680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44caa1e-a782-4ab2-923d-fa4ee8ec686e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:53.639515 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ef62f-4f8d-4d45-893b-9c780fe73ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:53.640131 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ef62f-4f8d-4d45-893b-9c780fe73ccd None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:53.640417 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ef62f-4f8d-4d45-893b-9c780fe73ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:53.640686 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ef62f-4f8d-4d45-893b-9c780fe73ccd None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:53.641886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ef62f-4f8d-4d45-893b-9c780fe73ccd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:53.641886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ef62f-4f8d-4d45-893b-9c780fe73ccd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:53.643005 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ef62f-4f8d-4d45-893b-9c780fe73ccd None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:53.654702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 632/1266] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:50:53.661097 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e5f36-21d1-4119-8528-490ad74dd252 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:53.757296 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e5f36-21d1-4119-8528-490ad74dd252 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:53.776500 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:53.790882 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e78a87-97b0-433c-96bb-029e15c6ab93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:53.791502 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e78a87-97b0-433c-96bb-029e15c6ab93 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:53.791780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e78a87-97b0-433c-96bb-029e15c6ab93 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:53.792049 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e78a87-97b0-433c-96bb-029e15c6ab93 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:53.799239 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e78a87-97b0-433c-96bb-029e15c6ab93 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:53.800885 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e78a87-97b0-433c-96bb-029e15c6ab93 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:53.811483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-90e78a87-97b0-433c-96bb-029e15c6ab93 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 20:50:53.811483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:50:53.811483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:50:53.811483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:50:53.811483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:50:53.811483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:50:53.811483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:50:53.811483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:50:53.811483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:50:53.811483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:50:53.811483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:50:53.811483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:50:53.811483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:50:53.811483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:50:53.811483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:50:53.811483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:50:53.812645 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:50:53.812645 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:50:53.812645 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:50:53.812645 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:50:53.812645 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:50:53.812645 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:50:53.812645 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:50:53.812645 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:50:53.812645 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:50:53.812645 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:50:53.812645 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:50:53.812645 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:50:53.812645 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:50:53.812645 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:50:53.812645 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:50:53.812645 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:50:53.812645 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:50:53.812645 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:50:53.813744 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:50:53.813744 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:50:53.813744 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:50:53.813744 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:50:53.813744 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:50:53.813744 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:50:53.813744 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:50:53.813744 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:50:53.813744 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 20:50:53.813744 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:50:53.813744 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 635/1267] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:50:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:50:53.818836 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ef62f-4f8d-4d45-893b-9c780fe73ccd None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:53.917833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ef62f-4f8d-4d45-893b-9c780fe73ccd None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:53.951454 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcaf619-ffd0-48e6-a201-cf8e303814f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:53.952047 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcaf619-ffd0-48e6-a201-cf8e303814f2 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:53.952356 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcaf619-ffd0-48e6-a201-cf8e303814f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:53.952627 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcaf619-ffd0-48e6-a201-cf8e303814f2 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:53.953341 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcaf619-ffd0-48e6-a201-cf8e303814f2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:53.953646 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcaf619-ffd0-48e6-a201-cf8e303814f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:53.955066 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcaf619-ffd0-48e6-a201-cf8e303814f2 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:53.965215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:53.975253 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 633/1268] 10.222.0.48 () {56 vars in 1143 bytes} [Fri Sep 9 20:50:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:50:53.981123 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e78a87-97b0-433c-96bb-029e15c6ab93 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:54.081144 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e78a87-97b0-433c-96bb-029e15c6ab93 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:54.113049 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e825e12-9dd8-441c-a481-e961542ea9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:54.113663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e825e12-9dd8-441c-a481-e961542ea9fb None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:54.113938 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e825e12-9dd8-441c-a481-e961542ea9fb None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:54.114206 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e825e12-9dd8-441c-a481-e961542ea9fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:54.124604 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e825e12-9dd8-441c-a481-e961542ea9fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:54.126020 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e825e12-9dd8-441c-a481-e961542ea9fb None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:54.140317 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-8e825e12-9dd8-441c-a481-e961542ea9fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:50:54.140317 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:50:54.140317 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:50:54.140317 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:50:54.140317 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:50:54.140317 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:50:54.140317 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:50:54.140317 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:50:54.140317 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:50:54.140317 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:50:54.140317 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:50:54.140317 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:50:54.140317 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:50:54.140317 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:50:54.140317 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:50:54.140317 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:50:54.141318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:50:54.141318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:50:54.141318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:50:54.141318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:50:54.141318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:50:54.141318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:50:54.141318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:50:54.141318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:50:54.141318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:50:54.141318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:50:54.141318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:50:54.141318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:50:54.141318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:50:54.141318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:50:54.141318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:50:54.141318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:50:54.141318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:50:54.142241 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:50:54.142241 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:50:54.142241 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:50:54.142241 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 636/1269] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:50:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:50:54.147850 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcaf619-ffd0-48e6-a201-cf8e303814f2 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:54.245559 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcaf619-ffd0-48e6-a201-cf8e303814f2 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:54.293219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8dc19b-ca9c-4cf5-a929-557cb198a18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:54.293826 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8dc19b-ca9c-4cf5-a929-557cb198a18c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:54.294104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8dc19b-ca9c-4cf5-a929-557cb198a18c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:54.294373 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8dc19b-ca9c-4cf5-a929-557cb198a18c None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:54.295010 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8dc19b-ca9c-4cf5-a929-557cb198a18c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:54.295311 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8dc19b-ca9c-4cf5-a929-557cb198a18c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:54.296707 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8dc19b-ca9c-4cf5-a929-557cb198a18c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:54.308408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 634/1270] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:50:54.314112 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e825e12-9dd8-441c-a481-e961542ea9fb None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:54.346513 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:54.413090 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e825e12-9dd8-441c-a481-e961542ea9fb None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:54.446719 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e177ff1c-fd6d-4c44-b8bf-d940c5f4ad9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:54.447341 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177ff1c-fd6d-4c44-b8bf-d940c5f4ad9e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:54.447618 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177ff1c-fd6d-4c44-b8bf-d940c5f4ad9e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:54.447888 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177ff1c-fd6d-4c44-b8bf-d940c5f4ad9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:54.454509 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e177ff1c-fd6d-4c44-b8bf-d940c5f4ad9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:54.456016 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e177ff1c-fd6d-4c44-b8bf-d940c5f4ad9e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:54.467004 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-e177ff1c-fd6d-4c44-b8bf-d940c5f4ad9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:50:54.467004 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:50:54.467004 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:50:54.467004 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:50:54.467004 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:50:54.467004 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:50:54.467004 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:50:54.467004 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:50:54.467004 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:50:54.467004 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:50:54.467004 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:50:54.467004 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:50:54.467004 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:50:54.467004 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:50:54.467004 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:50:54.468024 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:50:54.468024 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:50:54.468024 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:50:54.468024 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:50:54.468024 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:50:54.468024 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:50:54.468024 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:50:54.468024 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:50:54.468024 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:50:54.468024 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:50:54.468024 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:50:54.468024 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:50:54.468024 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:50:54.468024 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:50:54.468024 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:50:54.468024 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:50:54.468024 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:50:54.469118 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:50:54.469118 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:50:54.469118 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:50:54.469118 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:50:54.469118 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:50:54.469118 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:50:54.469118 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:50:54.469118 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:50:54.469118 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 637/1271] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:50:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:50:54.473975 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8dc19b-ca9c-4cf5-a929-557cb198a18c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:54.532477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:54.577924 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8dc19b-ca9c-4cf5-a929-557cb198a18c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:54.612534 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec040b4-12d4-4d1a-891b-4499a0452175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:54.613184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec040b4-12d4-4d1a-891b-4499a0452175 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:54.613493 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec040b4-12d4-4d1a-891b-4499a0452175 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:54.613766 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec040b4-12d4-4d1a-891b-4499a0452175 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:54.614460 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec040b4-12d4-4d1a-891b-4499a0452175 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:54.614767 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec040b4-12d4-4d1a-891b-4499a0452175 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:54.616223 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec040b4-12d4-4d1a-891b-4499a0452175 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:54.627866 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 635/1272] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:50:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:50:54.633885 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e177ff1c-fd6d-4c44-b8bf-d940c5f4ad9e None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:54.723493 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:54.732145 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e177ff1c-fd6d-4c44-b8bf-d940c5f4ad9e None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:54.764624 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ffd367-689a-451a-8e6f-a2e6dbd9a565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:54.765267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffd367-689a-451a-8e6f-a2e6dbd9a565 None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:54.765541 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffd367-689a-451a-8e6f-a2e6dbd9a565 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:54.765807 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffd367-689a-451a-8e6f-a2e6dbd9a565 None admin] PATH_INFO: `/v3/projects/91319aa782334fb9930770029b74e84a/users/2a4f6de2065d4fbea3fa141da760f708/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:54.767655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ffd367-689a-451a-8e6f-a2e6dbd9a565 None admin] RBAC: Authorizing `identity:create_grant(project_id=91319aa782334fb9930770029b74e84a, user_id=2a4f6de2065d4fbea3fa141da760f708, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:54.769090 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ffd367-689a-451a-8e6f-a2e6dbd9a565 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:54.832802 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 638/1273] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Sep 9 20:50:54 2022] PUT /identity/v3/projects/91319aa782334fb9930770029b74e84a/users/2a4f6de2065d4fbea3fa141da760f708/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 20:50:56.964572 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fa9ba-36d0-4fa1-80de-db20e0ea747c None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:56.965108 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fa9ba-36d0-4fa1-80de-db20e0ea747c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:56.965108 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fa9ba-36d0-4fa1-80de-db20e0ea747c None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:56.966422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 636/1274] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:50:56.971694 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d157ad-9a47-4006-a76a-daa4e730c61a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:56.972141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d157ad-9a47-4006-a76a-daa4e730c61a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:56.972265 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d157ad-9a47-4006-a76a-daa4e730c61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:56.990429 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-55d157ad-9a47-4006-a76a-daa4e730c61a None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:50:56.991596 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-55d157ad-9a47-4006-a76a-daa4e730c61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:50:57.057024 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 639/1275] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:50:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:50:57.326620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0fa9ba-36d0-4fa1-80de-db20e0ea747c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:57.425827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0fa9ba-36d0-4fa1-80de-db20e0ea747c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:57.443607 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:57.460773 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d153f4-7395-4fc0-92c1-d489382df766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:57.461392 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d153f4-7395-4fc0-92c1-d489382df766 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:57.461667 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d153f4-7395-4fc0-92c1-d489382df766 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:57.461937 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d153f4-7395-4fc0-92c1-d489382df766 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:57.471494 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d153f4-7395-4fc0-92c1-d489382df766 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 20:50:57.476297 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-f6d153f4-7395-4fc0-92c1-d489382df766 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:50:57.476297 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:50:57.476297 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:50:57.476297 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:50:57.476297 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:50:57.476297 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:50:57.476297 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:50:57.476297 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:50:57.476297 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:50:57.476297 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:50:57.476297 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:50:57.476297 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:50:57.476297 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 20:50:57.476297 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 20:50:57.476297 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 20:50:57.476297 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 20:50:57.478595 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 20:50:57.478595 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 20:50:57.478595 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 20:50:57.478595 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 20:50:57.478595 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:50:57.478595 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:50:57.478595 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:50:57.478595 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:50:57.478595 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:50:57.478595 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:50:57.478595 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:50:57.478595 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:50:57.478595 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:50:57.478595 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:50:57.478595 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:50:57.478595 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:50:57.479643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:50:57.479643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:50:57.479643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:50:57.479643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:50:57.479643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 20:50:57.479643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 20:50:57.479643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 20:50:57.479643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 20:50:57.479643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 20:50:57.479643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 20:50:57.479643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 20:50:57.479643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:50:57.479643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 637/1276] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Sep 9 20:50:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:50:57.483532 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d157ad-9a47-4006-a76a-daa4e730c61a None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:57.562164 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:57.580347 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d157ad-9a47-4006-a76a-daa4e730c61a None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:57.615747 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2901719d-be72-42c8-845c-afaedf6c1618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:57.616379 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901719d-be72-42c8-845c-afaedf6c1618 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:57.616657 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901719d-be72-42c8-845c-afaedf6c1618 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:57.616925 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901719d-be72-42c8-845c-afaedf6c1618 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:57.617588 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2901719d-be72-42c8-845c-afaedf6c1618 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:57.617884 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2901719d-be72-42c8-845c-afaedf6c1618 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:57.619235 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2901719d-be72-42c8-845c-afaedf6c1618 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:57.627321 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 640/1277] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Sep 9 20:50:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:50:57.633376 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d153f4-7395-4fc0-92c1-d489382df766 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:57.730374 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d153f4-7395-4fc0-92c1-d489382df766 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:57.763733 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc845b-e3db-4a63-a591-85a1e9aa36f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:57.764384 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc845b-e3db-4a63-a591-85a1e9aa36f7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:57.764667 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc845b-e3db-4a63-a591-85a1e9aa36f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:57.764942 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc845b-e3db-4a63-a591-85a1e9aa36f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:57.774659 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc845b-e3db-4a63-a591-85a1e9aa36f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:57.776122 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc845b-e3db-4a63-a591-85a1e9aa36f7 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:57.790844 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-dacc845b-e3db-4a63-a591-85a1e9aa36f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:50:57.790844 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:50:57.790844 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:50:57.790844 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:50:57.790844 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:50:57.790844 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:50:57.790844 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:50:57.790844 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:50:57.790844 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:50:57.790844 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:50:57.790844 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:50:57.790844 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:50:57.790844 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:50:57.790844 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:50:57.790844 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:50:57.790844 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:50:57.791991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:50:57.791991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:50:57.791991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:50:57.791991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:50:57.791991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:50:57.791991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:50:57.791991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:50:57.791991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:50:57.791991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:50:57.791991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:50:57.791991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:50:57.791991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:50:57.791991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:50:57.791991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:50:57.791991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:50:57.791991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:50:57.791991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:50:57.793161 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:50:57.793161 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:50:57.793161 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:50:57.793161 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 638/1278] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:50:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:50:57.797956 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2901719d-be72-42c8-845c-afaedf6c1618 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:57.895397 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2901719d-be72-42c8-845c-afaedf6c1618 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:57.928535 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792577f-d04b-40b9-8d4b-0cf65455ffd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:57.929128 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792577f-d04b-40b9-8d4b-0cf65455ffd9 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:57.929405 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792577f-d04b-40b9-8d4b-0cf65455ffd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:57.929671 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792577f-d04b-40b9-8d4b-0cf65455ffd9 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:57.930303 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8792577f-d04b-40b9-8d4b-0cf65455ffd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:57.930604 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8792577f-d04b-40b9-8d4b-0cf65455ffd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:57.931946 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8792577f-d04b-40b9-8d4b-0cf65455ffd9 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:57.944243 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 641/1279] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:50:57.950085 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc845b-e3db-4a63-a591-85a1e9aa36f7 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:57.986916 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:58.054518 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc845b-e3db-4a63-a591-85a1e9aa36f7 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:58.087765 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338d8a2-a72a-4db2-ad38-f740d6a681b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:58.088424 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338d8a2-a72a-4db2-ad38-f740d6a681b3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:58.088704 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338d8a2-a72a-4db2-ad38-f740d6a681b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:58.088977 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338d8a2-a72a-4db2-ad38-f740d6a681b3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:58.096793 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f338d8a2-a72a-4db2-ad38-f740d6a681b3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:58.098235 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f338d8a2-a72a-4db2-ad38-f740d6a681b3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:58.109226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-f338d8a2-a72a-4db2-ad38-f740d6a681b3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 20:50:58.109226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:50:58.109226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:50:58.109226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:50:58.109226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:50:58.109226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:50:58.109226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:50:58.109226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:50:58.109226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:50:58.109226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:50:58.109226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:50:58.109226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:50:58.109226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 20:50:58.109226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 20:50:58.109226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 20:50:58.109226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 20:50:58.110218 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:50:58.110218 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:50:58.110218 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 20:50:58.110218 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:50:58.110218 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 20:50:58.110218 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 20:50:58.110218 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:50:58.110218 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:50:58.110218 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:50:58.110218 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:50:58.110218 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:50:58.110218 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:50:58.110218 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:50:58.110218 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:50:58.110218 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:50:58.110218 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:50:58.110218 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:50:58.110218 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:50:58.111272 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:50:58.111272 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:50:58.111272 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 20:50:58.111272 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 20:50:58.111272 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 20:50:58.111272 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 20:50:58.111272 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 20:50:58.111272 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 20:50:58.111272 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 20:50:58.111272 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:50:58.111272 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 639/1280] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:50:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:50:58.116186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792577f-d04b-40b9-8d4b-0cf65455ffd9 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:58.119868 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:58.210676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792577f-d04b-40b9-8d4b-0cf65455ffd9 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:58.248783 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01732952-bf2d-415a-a33b-063445da0d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:58.249397 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01732952-bf2d-415a-a33b-063445da0d1a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:58.249676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01732952-bf2d-415a-a33b-063445da0d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:58.249944 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01732952-bf2d-415a-a33b-063445da0d1a None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:58.250679 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01732952-bf2d-415a-a33b-063445da0d1a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:58.251006 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01732952-bf2d-415a-a33b-063445da0d1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:58.252436 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01732952-bf2d-415a-a33b-063445da0d1a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:58.270127 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 642/1281] 10.222.0.48 () {56 vars in 1143 bytes} [Fri Sep 9 20:50:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:50:58.276019 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338d8a2-a72a-4db2-ad38-f740d6a681b3 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:58.365270 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:58.374799 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338d8a2-a72a-4db2-ad38-f740d6a681b3 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:58.407766 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359ccbe-f797-4240-a94e-daedd06e828d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:58.408424 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359ccbe-f797-4240-a94e-daedd06e828d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:58.408702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359ccbe-f797-4240-a94e-daedd06e828d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:58.408972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359ccbe-f797-4240-a94e-daedd06e828d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:58.474264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f359ccbe-f797-4240-a94e-daedd06e828d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:58.475726 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f359ccbe-f797-4240-a94e-daedd06e828d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:58.566760 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:58.577043 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 640/1282] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Sep 9 20:50:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 20:50:58.583502 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01732952-bf2d-415a-a33b-063445da0d1a None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:58.642714 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:58.679750 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01732952-bf2d-415a-a33b-063445da0d1a None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:58.713573 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69338ce8-781f-4e9f-a0a7-80025224b3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:58.714196 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69338ce8-781f-4e9f-a0a7-80025224b3ee None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:58.714475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69338ce8-781f-4e9f-a0a7-80025224b3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:58.714746 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69338ce8-781f-4e9f-a0a7-80025224b3ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:58.724582 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69338ce8-781f-4e9f-a0a7-80025224b3ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:58.725960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69338ce8-781f-4e9f-a0a7-80025224b3ee None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:58.740644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-69338ce8-781f-4e9f-a0a7-80025224b3ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:50:58.740644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:50:58.740644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:50:58.740644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:50:58.740644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:50:58.740644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:50:58.740644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:50:58.740644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:50:58.740644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:50:58.740644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:50:58.740644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:50:58.740644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:50:58.740644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 20:50:58.740644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 20:50:58.740644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 20:50:58.740644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 20:50:58.745471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:50:58.745471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:50:58.745471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:50:58.745471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:50:58.745471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:50:58.745471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:50:58.745471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:50:58.745471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:50:58.745471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:50:58.745471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:50:58.745471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:50:58.745471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:50:58.745471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:50:58.745471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:50:58.745471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:50:58.745471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:50:58.745471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 20:50:58.746574 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 20:50:58.746574 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 20:50:58.746574 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:50:58.746574 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 643/1283] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:50:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:50:58.747616 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359ccbe-f797-4240-a94e-daedd06e828d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:58.851162 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359ccbe-f797-4240-a94e-daedd06e828d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:58.886192 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab08de-b5ca-4dba-ab7c-7af81cc04751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:58.886794 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab08de-b5ca-4dba-ab7c-7af81cc04751 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:58.887072 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab08de-b5ca-4dba-ab7c-7af81cc04751 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:58.887341 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab08de-b5ca-4dba-ab7c-7af81cc04751 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:58.888007 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab08de-b5ca-4dba-ab7c-7af81cc04751 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:58.888348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab08de-b5ca-4dba-ab7c-7af81cc04751 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:58.889722 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab08de-b5ca-4dba-ab7c-7af81cc04751 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:58.901223 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 641/1284] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Sep 9 20:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 20:50:58.907103 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69338ce8-781f-4e9f-a0a7-80025224b3ee None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:59.005950 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69338ce8-781f-4e9f-a0a7-80025224b3ee None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:59.019190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:59.038495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406ca9b-8c55-421b-98ee-7b76bf3a1f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:59.039118 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406ca9b-8c55-421b-98ee-7b76bf3a1f60 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:59.039394 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406ca9b-8c55-421b-98ee-7b76bf3a1f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:59.039663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406ca9b-8c55-421b-98ee-7b76bf3a1f60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:59.045988 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a406ca9b-8c55-421b-98ee-7b76bf3a1f60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:59.047473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a406ca9b-8c55-421b-98ee-7b76bf3a1f60 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:59.059854 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-a406ca9b-8c55-421b-98ee-7b76bf3a1f60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:50:59.059854 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:50:59.059854 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:50:59.059854 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:50:59.059854 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:50:59.059854 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:50:59.059854 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:50:59.059854 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:50:59.059854 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:50:59.059854 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:50:59.059854 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:50:59.059854 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:50:59.059854 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 20:50:59.059854 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 20:50:59.059854 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 20:50:59.060977 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 20:50:59.060977 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:50:59.060977 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:50:59.060977 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:50:59.060977 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:50:59.060977 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:50:59.060977 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:50:59.060977 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:50:59.060977 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 09 20:50:59.060977 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:50:59.060977 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:50:59.060977 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:50:59.060977 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:50:59.060977 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:50:59.060977 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:50:59.060977 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:50:59.060977 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:50:59.062267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 20:50:59.062267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 20:50:59.062267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 20:50:59.062267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 20:50:59.062267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 20:50:59.062267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 20:50:59.062267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 20:50:59.062267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 09 20:50:59.062267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 644/1285] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Sep 9 20:50:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:50:59.067568 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab08de-b5ca-4dba-ab7c-7af81cc04751 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:59.117408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:50:59.165808 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab08de-b5ca-4dba-ab7c-7af81cc04751 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:59.200147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1d980-9798-4d04-a8d0-88ef6a77be17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:59.200752 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1d980-9798-4d04-a8d0-88ef6a77be17 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:59.201034 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1d980-9798-4d04-a8d0-88ef6a77be17 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:59.201312 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1d980-9798-4d04-a8d0-88ef6a77be17 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:59.202000 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e1d980-9798-4d04-a8d0-88ef6a77be17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:59.202307 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e1d980-9798-4d04-a8d0-88ef6a77be17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:59.203737 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e1d980-9798-4d04-a8d0-88ef6a77be17 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:59.215617 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 642/1286] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Sep 9 20:50:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:50:59.222353 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406ca9b-8c55-421b-98ee-7b76bf3a1f60 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:50:59.318850 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406ca9b-8c55-421b-98ee-7b76bf3a1f60 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:50:59.350913 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c10856-7749-47c3-836b-19547755e6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:50:59.351500 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c10856-7749-47c3-836b-19547755e6e5 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:50:59.351795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c10856-7749-47c3-836b-19547755e6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:50:59.352042 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c10856-7749-47c3-836b-19547755e6e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:50:59.352783 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c10856-7749-47c3-836b-19547755e6e5 None admin] RBAC: Adding query filter params (role.id=fcbf519fbc84458f913671a5a1d0914c, scope.project.id=91319aa782334fb9930770029b74e84a, user.id=2a4f6de2065d4fbea3fa141da760f708) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:50:59.353084 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c10856-7749-47c3-836b-19547755e6e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:50:59.354489 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c10856-7749-47c3-836b-19547755e6e5 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:50:59.362773 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 645/1287] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Sep 9 20:50:59 2022] GET /identity/v3/role_assignments?user.id=2a4f6de2065d4fbea3fa141da760f708&scope.project.id=91319aa782334fb9930770029b74e84a&role.id=fcbf519fbc84458f913671a5a1d0914c => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:51:01.482789 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3302202-5f8c-4150-a6c5-9c539e2bcbd7 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:51:01.483326 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3302202-5f8c-4150-a6c5-9c539e2bcbd7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:51:01.483326 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3302202-5f8c-4150-a6c5-9c539e2bcbd7 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:51:01.484677 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 643/1288] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:51:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:51:01.489888 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1b4aa-a56d-4966-8f40-17ae16cb4cf3 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:51:01.490273 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1b4aa-a56d-4966-8f40-17ae16cb4cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:51:01.490446 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1b4aa-a56d-4966-8f40-17ae16cb4cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:51:01.509820 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-e1d1b4aa-a56d-4966-8f40-17ae16cb4cf3 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:51:01.511101 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e1d1b4aa-a56d-4966-8f40-17ae16cb4cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:51:01.540092 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:51:01.576229 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 646/1289] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:51:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:51:01.841372 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3302202-5f8c-4150-a6c5-9c539e2bcbd7 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:51:01.945153 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3302202-5f8c-4150-a6c5-9c539e2bcbd7 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:51:01.955384 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:51:01.977440 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2cb0ac-aa22-475c-8fdb-b3efa66e7dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:51:01.978525 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2cb0ac-aa22-475c-8fdb-b3efa66e7dcf None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:51:01.978798 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2cb0ac-aa22-475c-8fdb-b3efa66e7dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:51:01.979063 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2cb0ac-aa22-475c-8fdb-b3efa66e7dcf None admin] PATH_INFO: `/v3/services/placement` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:51:01.979708 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2cb0ac-aa22-475c-8fdb-b3efa66e7dcf None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:51:01.981131 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2cb0ac-aa22-475c-8fdb-b3efa66e7dcf None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:51:01.991241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-6a2cb0ac-aa22-475c-8fdb-b3efa66e7dcf None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 20:51:01.991241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:51:01.991241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:51:01.991241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:51:01.991241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:51:01.991241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:51:01.991241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:51:01.991241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:51:01.991241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 20:51:01.991241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:51:01.991241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:51:01.991241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:51:01.991241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:51:01.991241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:51:01.991241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:51:01.991241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:51:01.991241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:51:01.992421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:51:01.992421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:51:01.992421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:51:01.992421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:51:01.992421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:51:01.992421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:51:01.992421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:51:01.992421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:51:01.992421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:51:01.992421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:51:01.992421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:51:01.992421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:51:01.992421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:51:01.992421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:51:01.992421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:51:01.992421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:51:01.993699 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:51:01.993699 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:51:01.993699 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:51:01.993699 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:51:01.993699 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:51:01.993699 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:51:01.993699 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:51:01.993699 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:51:01.993699 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:51:01.993699 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:51:01.993699 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:51:01.993699 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:51:01.993699 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 20:51:01.993699 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:51:01.993699 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 644/1290] 10.222.0.48 () {56 vars in 1117 bytes} [Fri Sep 9 20:51:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:51:01.998481 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d1b4aa-a56d-4966-8f40-17ae16cb4cf3 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:51:02.095379 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d1b4aa-a56d-4966-8f40-17ae16cb4cf3 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:51:02.130364 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68434791-76aa-4f5c-9556-d1e49e086653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:51:02.130955 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68434791-76aa-4f5c-9556-d1e49e086653 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:51:02.131234 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68434791-76aa-4f5c-9556-d1e49e086653 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:51:02.131556 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68434791-76aa-4f5c-9556-d1e49e086653 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:51:02.132212 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68434791-76aa-4f5c-9556-d1e49e086653 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:51:02.132520 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68434791-76aa-4f5c-9556-d1e49e086653 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:51:02.133877 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68434791-76aa-4f5c-9556-d1e49e086653 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:51:02.138696 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 647/1291] 10.222.0.48 () {56 vars in 1116 bytes} [Fri Sep 9 20:51:01 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:51:02.144654 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2cb0ac-aa22-475c-8fdb-b3efa66e7dcf None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:51:02.251900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2cb0ac-aa22-475c-8fdb-b3efa66e7dcf None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:51:02.285684 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b32662-2d2b-48be-952b-2ada62e02ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:51:02.286288 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b32662-2d2b-48be-952b-2ada62e02ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:51:02.286567 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b32662-2d2b-48be-952b-2ada62e02ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:51:02.286839 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b32662-2d2b-48be-952b-2ada62e02ec2 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:51:02.287463 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b32662-2d2b-48be-952b-2ada62e02ec2 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:51:02.287766 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b32662-2d2b-48be-952b-2ada62e02ec2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:51:02.289199 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b32662-2d2b-48be-952b-2ada62e02ec2 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:51:02.293654 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 645/1292] 10.222.0.48 () {56 vars in 1116 bytes} [Fri Sep 9 20:51:02 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:51:04.417664 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55cd343-0a38-47ee-818d-5b80771c175d None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:51:04.418186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55cd343-0a38-47ee-818d-5b80771c175d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:51:04.418186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55cd343-0a38-47ee-818d-5b80771c175d None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:51:04.419557 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 648/1293] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:51:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:51:04.424732 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c624f9-e8ac-4350-ba78-b8b21a066041 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:51:04.425217 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c624f9-e8ac-4350-ba78-b8b21a066041 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:51:04.425309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c624f9-e8ac-4350-ba78-b8b21a066041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:51:04.443832 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-37c624f9-e8ac-4350-ba78-b8b21a066041 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:51:04.445052 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-37c624f9-e8ac-4350-ba78-b8b21a066041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:51:04.474216 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:51:04.510431 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 646/1294] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:51:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:51:04.777496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55cd343-0a38-47ee-818d-5b80771c175d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:51:04.808184 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:51:04.877968 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55cd343-0a38-47ee-818d-5b80771c175d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:51:04.910751 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b520820-209b-4648-8f99-13d1c447d6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:51:04.911351 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b520820-209b-4648-8f99-13d1c447d6b1 None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:51:04.911628 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b520820-209b-4648-8f99-13d1c447d6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:51:04.911931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b520820-209b-4648-8f99-13d1c447d6b1 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:51:04.912593 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b520820-209b-4648-8f99-13d1c447d6b1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:51:04.913950 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b520820-209b-4648-8f99-13d1c447d6b1 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:51:05.015773 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 649/1295] 10.222.0.48 () {60 vars in 1141 bytes} [Fri Sep 9 20:51:04 2022] POST /identity/v3/services => generated 246 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 20:51:07.231905 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff27d41-6d9d-4b21-a316-5b98810f7b5d None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:51:07.232544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff27d41-6d9d-4b21-a316-5b98810f7b5d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:51:07.232544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff27d41-6d9d-4b21-a316-5b98810f7b5d None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:51:07.233825 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 647/1296] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:51:07.239600 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538704f0-668f-4d40-82b6-a79622c475b2 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:51:07.242939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538704f0-668f-4d40-82b6-a79622c475b2 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:51:07.242939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538704f0-668f-4d40-82b6-a79622c475b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:51:07.259424 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-538704f0-668f-4d40-82b6-a79622c475b2 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:51:07.260743 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-538704f0-668f-4d40-82b6-a79622c475b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:51:07.335645 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 650/1297] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:51:07 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:51:07.631494 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff27d41-6d9d-4b21-a316-5b98810f7b5d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:51:07.731984 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff27d41-6d9d-4b21-a316-5b98810f7b5d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:51:07.763975 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb55f85-a2c9-4aa0-84f3-31ab8449e0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:51:07.764615 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb55f85-a2c9-4aa0-84f3-31ab8449e0f9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:51:07.764892 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb55f85-a2c9-4aa0-84f3-31ab8449e0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:51:07.765162 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb55f85-a2c9-4aa0-84f3-31ab8449e0f9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:51:07.765735 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb55f85-a2c9-4aa0-84f3-31ab8449e0f9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:51:07.767092 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb55f85-a2c9-4aa0-84f3-31ab8449e0f9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:51:07.776013 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-bbb55f85-a2c9-4aa0-84f3-31ab8449e0f9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 20:51:07.776013 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:51:07.776013 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:51:07.776013 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:51:07.776013 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:51:07.776013 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:51:07.776013 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:51:07.776013 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:51:07.776013 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 20:51:07.776013 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:51:07.776013 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:51:07.776013 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:51:07.776013 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:51:07.776013 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:51:07.776013 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:51:07.776013 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:51:07.776013 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:51:07.777283 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:51:07.777283 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:51:07.777283 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:51:07.777283 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:51:07.777283 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:51:07.777283 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:51:07.777283 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:51:07.777283 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:51:07.777283 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:51:07.777283 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:51:07.777283 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:51:07.777283 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:51:07.777283 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:51:07.777283 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:51:07.777283 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:51:07.777283 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:51:07.778416 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:51:07.778416 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:51:07.778416 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:51:07.778416 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:51:07.778416 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:51:07.778416 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:51:07.778416 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:51:07.778416 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:51:07.778416 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:51:07.778416 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:51:07.778416 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:51:07.778416 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:51:07.778416 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 20:51:07.778416 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:51:07.778416 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 648/1298] 10.222.0.48 () {56 vars in 1117 bytes} [Fri Sep 9 20:51:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:51:07.783274 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538704f0-668f-4d40-82b6-a79622c475b2 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:51:07.801763 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:51:07.880062 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538704f0-668f-4d40-82b6-a79622c475b2 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:51:07.914790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc3525-dcfb-42d5-bf52-c4469bfe157a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:51:07.915484 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc3525-dcfb-42d5-bf52-c4469bfe157a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:51:07.915767 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc3525-dcfb-42d5-bf52-c4469bfe157a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:51:07.916039 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc3525-dcfb-42d5-bf52-c4469bfe157a None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:51:07.916836 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bc3525-dcfb-42d5-bf52-c4469bfe157a None admin] RBAC: Adding query filter params (name=placement) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:51:07.917148 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bc3525-dcfb-42d5-bf52-c4469bfe157a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:51:07.918565 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bc3525-dcfb-42d5-bf52-c4469bfe157a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:51:07.923981 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 651/1299] 10.222.0.48 () {56 vars in 1116 bytes} [Fri Sep 9 20:51:07 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:51:07.930376 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb55f85-a2c9-4aa0-84f3-31ab8449e0f9 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:51:07.961808 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:51:08.029424 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb55f85-a2c9-4aa0-84f3-31ab8449e0f9 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:51:08.061701 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720caf21-9b98-4fb8-b04b-a3cfd032ad8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:51:08.062291 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720caf21-9b98-4fb8-b04b-a3cfd032ad8b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:51:08.062564 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720caf21-9b98-4fb8-b04b-a3cfd032ad8b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:51:08.062831 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720caf21-9b98-4fb8-b04b-a3cfd032ad8b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:51:08.063521 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720caf21-9b98-4fb8-b04b-a3cfd032ad8b None admin] RBAC: Adding query filter params (interface=public, service_id=a531aa2ac10943c7922cd43f05f48d42, region_id=RegionOne) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:51:08.063827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720caf21-9b98-4fb8-b04b-a3cfd032ad8b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:51:08.066215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720caf21-9b98-4fb8-b04b-a3cfd032ad8b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:51:08.071031 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 649/1300] 10.222.0.48 () {56 vars in 1251 bytes} [Fri Sep 9 20:51:07 2022] GET /identity/v3/endpoints?service_id=a531aa2ac10943c7922cd43f05f48d42&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:51:08.076774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc3525-dcfb-42d5-bf52-c4469bfe157a None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:51:08.155513 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:51:08.173046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc3525-dcfb-42d5-bf52-c4469bfe157a None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:51:08.206365 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c457273-c643-437f-b285-d05a1a1a6204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:51:08.206939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c457273-c643-437f-b285-d05a1a1a6204 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:51:08.207218 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c457273-c643-437f-b285-d05a1a1a6204 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:51:08.207483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c457273-c643-437f-b285-d05a1a1a6204 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:51:08.208104 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c457273-c643-437f-b285-d05a1a1a6204 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:51:08.209436 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c457273-c643-437f-b285-d05a1a1a6204 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:51:08.215872 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 652/1301] 10.222.0.48 () {56 vars in 1087 bytes} [Fri Sep 9 20:51:08 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:51:10.357745 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c3159-9f98-431e-bd63-9a006661fe84 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:51:10.358263 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c3159-9f98-431e-bd63-9a006661fe84 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:51:10.358263 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c3159-9f98-431e-bd63-9a006661fe84 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:51:10.359518 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 650/1302] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:51:10.364706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c7d12-39aa-4eb7-994d-dedfb606b78d None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:51:10.365095 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c7d12-39aa-4eb7-994d-dedfb606b78d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:51:10.365250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c7d12-39aa-4eb7-994d-dedfb606b78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:51:10.384298 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-c93c7d12-39aa-4eb7-994d-dedfb606b78d None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:51:10.386898 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c93c7d12-39aa-4eb7-994d-dedfb606b78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:51:10.450564 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 653/1303] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:51:10 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 20:51:10.725937 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c3159-9f98-431e-bd63-9a006661fe84 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:51:10.819665 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:51:10.826230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c3159-9f98-431e-bd63-9a006661fe84 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:51:10.858323 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7996da2-f524-4768-b498-34e01f0f7798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:51:10.858936 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7996da2-f524-4768-b498-34e01f0f7798 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:51:10.859261 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7996da2-f524-4768-b498-34e01f0f7798 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:51:10.859536 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7996da2-f524-4768-b498-34e01f0f7798 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:51:10.860137 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7996da2-f524-4768-b498-34e01f0f7798 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:51:10.862031 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7996da2-f524-4768-b498-34e01f0f7798 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:51:10.872050 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-a7996da2-f524-4768-b498-34e01f0f7798 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 20:51:10.872050 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:51:10.872050 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 20:51:10.872050 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 20:51:10.872050 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 20:51:10.872050 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 20:51:10.872050 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 20:51:10.872050 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:51:10.872050 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 20:51:10.872050 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:51:10.872050 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 20:51:10.872050 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:51:10.872050 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 20:51:10.872050 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 20:51:10.872050 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 20:51:10.872050 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 20:51:10.872050 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 20:51:10.895585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 20:51:10.895585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 20:51:10.895585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 20:51:10.895585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 20:51:10.895585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 20:51:10.895585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 20:51:10.895585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 20:51:10.895585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 20:51:10.895585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 20:51:10.895585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 20:51:10.895585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 20:51:10.895585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 20:51:10.895585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 20:51:10.895585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 20:51:10.895585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 20:51:10.895585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 20:51:10.896643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 20:51:10.896643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 09 20:51:10.896643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 20:51:10.896643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 09 20:51:10.896643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 20:51:10.896643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 20:51:10.896643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 20:51:10.896643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 09 20:51:10.896643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 20:51:10.896643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 09 20:51:10.896643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 20:51:10.896643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 20:51:10.896643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 20:51:10.896643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 09 20:51:10.896643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 651/1304] 10.222.0.48 () {56 vars in 1117 bytes} [Fri Sep 9 20:51:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 20:51:10.897577 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c7d12-39aa-4eb7-994d-dedfb606b78d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:51:10.959299 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:51:10.980766 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c7d12-39aa-4eb7-994d-dedfb606b78d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:51:11.015327 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399fb10-a186-498b-836d-8162f80c8ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:51:11.015930 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399fb10-a186-498b-836d-8162f80c8ff4 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:51:11.016237 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399fb10-a186-498b-836d-8162f80c8ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:51:11.016508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399fb10-a186-498b-836d-8162f80c8ff4 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:51:11.017132 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e399fb10-a186-498b-836d-8162f80c8ff4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 20:51:11.017433 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e399fb10-a186-498b-836d-8162f80c8ff4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:51:11.018810 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e399fb10-a186-498b-836d-8162f80c8ff4 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:51:11.023807 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 654/1305] 10.222.0.48 () {56 vars in 1116 bytes} [Fri Sep 9 20:51:10 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 20:51:11.029646 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7996da2-f524-4768-b498-34e01f0f7798 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:51:11.128030 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7996da2-f524-4768-b498-34e01f0f7798 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:51:11.160533 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67542a-83c2-4c73-9c1b-d13351321193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:51:11.161130 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67542a-83c2-4c73-9c1b-d13351321193 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:51:11.161408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67542a-83c2-4c73-9c1b-d13351321193 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:51:11.161677 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67542a-83c2-4c73-9c1b-d13351321193 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:51:11.162332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a67542a-83c2-4c73-9c1b-d13351321193 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:51:11.163690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a67542a-83c2-4c73-9c1b-d13351321193 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:51:11.226685 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 652/1306] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 20:51:11 2022] POST /identity/v3/endpoints => generated 331 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 20:51:21.020828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=30 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:51:21.222975 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:56:51.601473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5d17a-b4ac-49b4-9841-0fea77909e87 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:56:51.601935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5d17a-b4ac-49b4-9841-0fea77909e87 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:56:51.601935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5d17a-b4ac-49b4-9841-0fea77909e87 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:56:51.603456 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 655/1307] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:56:51.609057 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a42679-d7b3-40b2-bc6f-d2521f3c9bb0 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:56:51.609462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a42679-d7b3-40b2-bc6f-d2521f3c9bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:56:51.609614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a42679-d7b3-40b2-bc6f-d2521f3c9bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:56:51.777969 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-41a42679-d7b3-40b2-bc6f-d2521f3c9bb0 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:56:51.781673 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-41a42679-d7b3-40b2-bc6f-d2521f3c9bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:56:51.952119 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 653/1308] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:56:52.254971 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a65980-89ac-4996-a9f6-aaf7dcd64011 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:56:52.255617 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a65980-89ac-4996-a9f6-aaf7dcd64011 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:56:52.256169 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a65980-89ac-4996-a9f6-aaf7dcd64011 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:56:52.257725 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 656/1309] 10.222.0.48 () {54 vars in 965 bytes} [Fri Sep 9 20:56:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:56:52.265303 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d66f6-ba86-4e8d-9a09-02371df5c3cb None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:56:52.265303 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d66f6-ba86-4e8d-9a09-02371df5c3cb None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:56:52.265303 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d66f6-ba86-4e8d-9a09-02371df5c3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:56:52.357530 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-0a8d66f6-ba86-4e8d-9a09-02371df5c3cb None None] MFA Rules not processed for user `8f5dfac06ac9492a86503ba133c638a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:56:52.358784 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0a8d66f6-ba86-4e8d-9a09-02371df5c3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:56:52.385135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:56:52.455457 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 654/1310] 10.222.0.48 () {58 vars in 1062 bytes} [Fri Sep 9 20:56:52 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:56:52.464719 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a65980-89ac-4996-a9f6-aaf7dcd64011 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:56:52.466902 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e6a65980-89ac-4996-a9f6-aaf7dcd64011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:56:52.598248 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a65980-89ac-4996-a9f6-aaf7dcd64011 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:56:52.686365 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd325954-7948-4104-9b48-7fea3d167454 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:56:52.687924 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd325954-7948-4104-9b48-7fea3d167454 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:56:52.688552 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd325954-7948-4104-9b48-7fea3d167454 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:56:52.689042 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd325954-7948-4104-9b48-7fea3d167454 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:56:52.691132 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-cd325954-7948-4104-9b48-7fea3d167454 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:56:52.763630 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:56:52.838118 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd325954-7948-4104-9b48-7fea3d167454 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:56:52.839859 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd325954-7948-4104-9b48-7fea3d167454 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:56:52.866527 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 657/1311] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:56:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:56:55.330457 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efaf09-1818-4e0d-b662-6a7f9400aab2 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:56:55.330973 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efaf09-1818-4e0d-b662-6a7f9400aab2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:56:55.330973 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efaf09-1818-4e0d-b662-6a7f9400aab2 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:56:55.332275 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 655/1312] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:56:55.337570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4cee4-a938-41f9-a49a-7ea32f81a554 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:56:55.338033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4cee4-a938-41f9-a49a-7ea32f81a554 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:56:55.338210 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4cee4-a938-41f9-a49a-7ea32f81a554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:56:55.420834 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-9cc4cee4-a938-41f9-a49a-7ea32f81a554 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:56:55.422032 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-9cc4cee4-a938-41f9-a49a-7ea32f81a554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:56:55.519243 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 658/1313] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:56:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:56:55.806990 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1efaf09-1818-4e0d-b662-6a7f9400aab2 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:56:55.808532 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c1efaf09-1818-4e0d-b662-6a7f9400aab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:56:55.912006 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1efaf09-1818-4e0d-b662-6a7f9400aab2 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:56:55.982850 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:56:55.989961 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c3fa8-e573-4e9b-89ab-2f3759a5ded8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:56:55.990589 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c3fa8-e573-4e9b-89ab-2f3759a5ded8 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:56:55.994449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c3fa8-e573-4e9b-89ab-2f3759a5ded8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:56:55.994449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c3fa8-e573-4e9b-89ab-2f3759a5ded8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:56:55.994449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-500c3fa8-e573-4e9b-89ab-2f3759a5ded8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:56:56.104354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c3fa8-e573-4e9b-89ab-2f3759a5ded8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:56:56.105736 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c3fa8-e573-4e9b-89ab-2f3759a5ded8 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:56:56.123331 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 656/1314] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:56:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:56:58.759779 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3616344-fba9-4619-9b3b-8943513d7f8c None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:56:58.760382 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3616344-fba9-4619-9b3b-8943513d7f8c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:56:58.760382 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3616344-fba9-4619-9b3b-8943513d7f8c None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:56:58.761683 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 659/1315] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:56:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:56:58.766853 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058dee7b-cb57-4bdd-89d3-de2eef41f4a2 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:56:58.767260 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058dee7b-cb57-4bdd-89d3-de2eef41f4a2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:56:58.767406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058dee7b-cb57-4bdd-89d3-de2eef41f4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:56:58.831832 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-058dee7b-cb57-4bdd-89d3-de2eef41f4a2 None None] MFA Rules not processed for user `2bca44a120f84b35b219ce7bed5a328a`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:56:58.833070 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-058dee7b-cb57-4bdd-89d3-de2eef41f4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:56:58.883604 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:56:58.927630 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 657/1316] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:56:58 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:56:59.222047 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3616344-fba9-4619-9b3b-8943513d7f8c None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:56:59.223596 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f3616344-fba9-4619-9b3b-8943513d7f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:56:59.233298 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:56:59.330295 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3616344-fba9-4619-9b3b-8943513d7f8c None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:56:59.402786 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a14db-82af-4244-b9e0-3138079bb3bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:56:59.403417 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a14db-82af-4244-b9e0-3138079bb3bc service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:56:59.403697 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a14db-82af-4244-b9e0-3138079bb3bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:56:59.403969 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a14db-82af-4244-b9e0-3138079bb3bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:56:59.405609 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ad8a14db-82af-4244-b9e0-3138079bb3bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:56:59.468806 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:56:59.513997 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a14db-82af-4244-b9e0-3138079bb3bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:56:59.515449 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a14db-82af-4244-b9e0-3138079bb3bc service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:56:59.534261 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 660/1317] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:56:59 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:57:02.423604 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33fd86-8c24-4224-a4cb-780f80819f4d None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:02.424148 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33fd86-8c24-4224-a4cb-780f80819f4d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:02.424269 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33fd86-8c24-4224-a4cb-780f80819f4d None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:02.425506 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 658/1318] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:57:02.430603 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31898ee5-d6ca-454f-8c8c-0796c1b3ef1a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:02.431024 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31898ee5-d6ca-454f-8c8c-0796c1b3ef1a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:02.431153 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31898ee5-d6ca-454f-8c8c-0796c1b3ef1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:02.495116 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-31898ee5-d6ca-454f-8c8c-0796c1b3ef1a None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:57:02.496338 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-31898ee5-d6ca-454f-8c8c-0796c1b3ef1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:02.594660 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 661/1319] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:57:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:57:02.875931 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33fd86-8c24-4224-a4cb-780f80819f4d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:57:02.877450 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2c33fd86-8c24-4224-a4cb-780f80819f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:02.977410 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33fd86-8c24-4224-a4cb-780f80819f4d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:57:03.050167 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e25af84-cf98-43bd-a14a-3932f442d831 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:57:03.050764 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25af84-cf98-43bd-a14a-3932f442d831 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:03.051039 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25af84-cf98-43bd-a14a-3932f442d831 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:03.051308 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25af84-cf98-43bd-a14a-3932f442d831 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:03.052902 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2e25af84-cf98-43bd-a14a-3932f442d831 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:03.059611 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:57:03.160799 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e25af84-cf98-43bd-a14a-3932f442d831 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:57:03.162238 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e25af84-cf98-43bd-a14a-3932f442d831 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:57:03.180164 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 659/1320] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:57:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:57:06.257292 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c9bb3-d0f4-418e-b5cb-689e86d3ba21 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:06.257831 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c9bb3-d0f4-418e-b5cb-689e86d3ba21 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:06.257831 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c9bb3-d0f4-418e-b5cb-689e86d3ba21 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:06.259145 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 662/1321] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:57:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:57:06.264351 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe2be7-091c-4658-976c-ee4f1eac5d42 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:06.264747 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe2be7-091c-4658-976c-ee4f1eac5d42 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:06.264851 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe2be7-091c-4658-976c-ee4f1eac5d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:06.331147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-0ebe2be7-091c-4658-976c-ee4f1eac5d42 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:57:06.332361 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0ebe2be7-091c-4658-976c-ee4f1eac5d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:06.389714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:57:06.429968 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 660/1322] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:57:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:57:06.709493 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c9bb3-d0f4-418e-b5cb-689e86d3ba21 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:57:06.711065 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3b6c9bb3-d0f4-418e-b5cb-689e86d3ba21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:06.731354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:57:06.814095 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c9bb3-d0f4-418e-b5cb-689e86d3ba21 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:57:06.886827 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c3ef1-8d2a-457a-b194-4759e28ba98b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:57:06.887929 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c3ef1-8d2a-457a-b194-4759e28ba98b service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:06.888247 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c3ef1-8d2a-457a-b194-4759e28ba98b service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:06.888520 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c3ef1-8d2a-457a-b194-4759e28ba98b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:06.890150 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8f0c3ef1-8d2a-457a-b194-4759e28ba98b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:06.963605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:57:07.001229 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0c3ef1-8d2a-457a-b194-4759e28ba98b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:57:07.002663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0c3ef1-8d2a-457a-b194-4759e28ba98b service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:57:07.024546 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 663/1323] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:57:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:57:09.206918 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4363580-e9af-4117-85e2-7b225132b1a3 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:09.207483 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4363580-e9af-4117-85e2-7b225132b1a3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:09.207483 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4363580-e9af-4117-85e2-7b225132b1a3 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:09.208857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 661/1324] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:57:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:57:09.214198 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03004b-046c-487b-a89c-035f79d2fc37 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:09.214611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03004b-046c-487b-a89c-035f79d2fc37 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:09.214748 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03004b-046c-487b-a89c-035f79d2fc37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:09.280157 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-cb03004b-046c-487b-a89c-035f79d2fc37 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:57:09.281304 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-cb03004b-046c-487b-a89c-035f79d2fc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:09.382452 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 664/1325] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:57:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:57:09.667065 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4363580-e9af-4117-85e2-7b225132b1a3 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:57:09.669390 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d4363580-e9af-4117-85e2-7b225132b1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:09.778527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4363580-e9af-4117-85e2-7b225132b1a3 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:57:09.852424 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b0899-4d3d-4f4c-9dea-53f212e8c483 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:57:09.852946 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b0899-4d3d-4f4c-9dea-53f212e8c483 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:09.853225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b0899-4d3d-4f4c-9dea-53f212e8c483 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:09.853495 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b0899-4d3d-4f4c-9dea-53f212e8c483 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:09.855074 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-682b0899-4d3d-4f4c-9dea-53f212e8c483 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:09.873222 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:57:09.964878 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b0899-4d3d-4f4c-9dea-53f212e8c483 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:57:09.966791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b0899-4d3d-4f4c-9dea-53f212e8c483 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:57:09.986053 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 662/1326] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:57:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:57:12.194903 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e55f5-6442-4ade-a1c3-345a4ee05858 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:12.195514 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e55f5-6442-4ade-a1c3-345a4ee05858 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:12.195589 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e55f5-6442-4ade-a1c3-345a4ee05858 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:12.196934 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 665/1327] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:57:12.203035 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56fe77-1d94-4b10-aa8d-aaf09b40398c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:12.203533 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56fe77-1d94-4b10-aa8d-aaf09b40398c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:12.203625 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56fe77-1d94-4b10-aa8d-aaf09b40398c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:12.275617 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-8b56fe77-1d94-4b10-aa8d-aaf09b40398c None None] MFA Rules not processed for user `2bca44a120f84b35b219ce7bed5a328a`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:57:12.276979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8b56fe77-1d94-4b10-aa8d-aaf09b40398c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:12.341632 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:57:12.374873 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 663/1328] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:57:12 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:57:12.677591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e55f5-6442-4ade-a1c3-345a4ee05858 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:57:12.679206 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-743e55f5-6442-4ade-a1c3-345a4ee05858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:12.710269 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:57:12.784722 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e55f5-6442-4ade-a1c3-345a4ee05858 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:57:12.861897 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea2385-4c43-404c-b514-8cc89b10ef35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:57:12.862511 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea2385-4c43-404c-b514-8cc89b10ef35 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:12.862789 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea2385-4c43-404c-b514-8cc89b10ef35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:12.863060 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea2385-4c43-404c-b514-8cc89b10ef35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:12.864691 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-30ea2385-4c43-404c-b514-8cc89b10ef35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:12.947188 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:57:12.980208 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ea2385-4c43-404c-b514-8cc89b10ef35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:57:12.982223 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ea2385-4c43-404c-b514-8cc89b10ef35 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:57:13.001842 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 666/1329] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:57:12 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:57:15.637405 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409b052-dbaf-4206-8151-6d46ab64fa6f None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:15.638030 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409b052-dbaf-4206-8151-6d46ab64fa6f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:15.638030 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409b052-dbaf-4206-8151-6d46ab64fa6f None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:15.639395 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 664/1330] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:57:15.644878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d553e-5a24-4de6-b83b-2210dca0ec84 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:15.645361 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d553e-5a24-4de6-b83b-2210dca0ec84 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:15.645470 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d553e-5a24-4de6-b83b-2210dca0ec84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:15.709689 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-bb7d553e-5a24-4de6-b83b-2210dca0ec84 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:57:15.710894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-bb7d553e-5a24-4de6-b83b-2210dca0ec84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:15.805518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 667/1331] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:57:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:57:16.088496 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409b052-dbaf-4206-8151-6d46ab64fa6f None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:57:16.090507 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a409b052-dbaf-4206-8151-6d46ab64fa6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:16.193715 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409b052-dbaf-4206-8151-6d46ab64fa6f None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:57:16.264899 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8656da24-ba06-4eaa-8249-6988fdea927a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:57:16.265487 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656da24-ba06-4eaa-8249-6988fdea927a service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:16.265761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656da24-ba06-4eaa-8249-6988fdea927a service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:16.266029 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656da24-ba06-4eaa-8249-6988fdea927a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:16.267594 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8656da24-ba06-4eaa-8249-6988fdea927a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:16.293733 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:57:16.373179 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8656da24-ba06-4eaa-8249-6988fdea927a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:57:16.374581 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8656da24-ba06-4eaa-8249-6988fdea927a service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:57:16.394031 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 665/1332] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:57:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:57:18.561603 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232f759-e8e0-43ff-a4e9-cbd9e51befad None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:18.562180 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232f759-e8e0-43ff-a4e9-cbd9e51befad None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:18.562180 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232f759-e8e0-43ff-a4e9-cbd9e51befad None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:18.563474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 668/1333] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:57:18.569115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd1301-6ff1-4894-9d11-e14dea232705 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:18.569553 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd1301-6ff1-4894-9d11-e14dea232705 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:18.569658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd1301-6ff1-4894-9d11-e14dea232705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:18.633393 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-68dd1301-6ff1-4894-9d11-e14dea232705 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:57:18.634577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-68dd1301-6ff1-4894-9d11-e14dea232705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:18.707971 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:57:18.730480 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 666/1334] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:57:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:57:19.013529 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232f759-e8e0-43ff-a4e9-cbd9e51befad None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:57:19.015064 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2232f759-e8e0-43ff-a4e9-cbd9e51befad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:19.054044 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:57:19.116125 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232f759-e8e0-43ff-a4e9-cbd9e51befad None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:57:19.188875 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c6c94-be2e-4317-826a-710ab9b5f382 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:57:19.189472 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c6c94-be2e-4317-826a-710ab9b5f382 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:19.189750 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c6c94-be2e-4317-826a-710ab9b5f382 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:19.190022 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c6c94-be2e-4317-826a-710ab9b5f382 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:19.191606 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-784c6c94-be2e-4317-826a-710ab9b5f382 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:19.276139 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:57:19.301230 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c6c94-be2e-4317-826a-710ab9b5f382 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:57:19.302654 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c6c94-be2e-4317-826a-710ab9b5f382 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:57:19.321588 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 669/1335] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:57:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:57:22.048436 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c77b2f-71fb-4833-b444-ca330e4aec9b None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:22.048946 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c77b2f-71fb-4833-b444-ca330e4aec9b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:22.049059 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c77b2f-71fb-4833-b444-ca330e4aec9b None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:22.050348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 667/1336] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:57:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:57:22.055630 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d1555-e1bd-41f7-ac22-e7e8f60d1fb3 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:22.056207 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d1555-e1bd-41f7-ac22-e7e8f60d1fb3 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:22.056207 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d1555-e1bd-41f7-ac22-e7e8f60d1fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:22.121603 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-fa1d1555-e1bd-41f7-ac22-e7e8f60d1fb3 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:57:22.122824 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-fa1d1555-e1bd-41f7-ac22-e7e8f60d1fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:22.217142 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 670/1337] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:57:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:57:22.501806 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c77b2f-71fb-4833-b444-ca330e4aec9b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:57:22.503342 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-86c77b2f-71fb-4833-b444-ca330e4aec9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:22.604203 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c77b2f-71fb-4833-b444-ca330e4aec9b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:57:22.676011 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9bb28-f985-48da-8857-274e8bc330ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:57:22.676639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9bb28-f985-48da-8857-274e8bc330ac service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:22.676919 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9bb28-f985-48da-8857-274e8bc330ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:22.677189 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9bb28-f985-48da-8857-274e8bc330ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:22.678762 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-28f9bb28-f985-48da-8857-274e8bc330ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:22.715245 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:57:22.786754 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f9bb28-f985-48da-8857-274e8bc330ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:57:22.788149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f9bb28-f985-48da-8857-274e8bc330ac service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:57:22.806866 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 668/1338] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:57:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:57:27.057807 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f0d21-485a-43f4-a5f9-badf1305ce87 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:27.058278 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f0d21-485a-43f4-a5f9-badf1305ce87 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:27.058925 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f0d21-485a-43f4-a5f9-badf1305ce87 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:27.060263 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 671/1339] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:57:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:57:27.065416 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5c8d7-7684-47f0-b762-04bae78af2de None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:27.065844 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5c8d7-7684-47f0-b762-04bae78af2de None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:27.065957 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5c8d7-7684-47f0-b762-04bae78af2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:27.131123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9fd5c8d7-7684-47f0-b762-04bae78af2de None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:57:27.132345 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9fd5c8d7-7684-47f0-b762-04bae78af2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:27.218489 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:57:27.229248 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 669/1340] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:57:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:57:27.511836 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f0d21-485a-43f4-a5f9-badf1305ce87 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:57:27.513590 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-017f0d21-485a-43f4-a5f9-badf1305ce87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:27.562175 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:57:27.616837 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f0d21-485a-43f4-a5f9-badf1305ce87 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:57:27.690640 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1646c4-409d-4a34-8cc0-893b2022571f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:57:27.691250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1646c4-409d-4a34-8cc0-893b2022571f service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:27.691542 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1646c4-409d-4a34-8cc0-893b2022571f service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:27.691796 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1646c4-409d-4a34-8cc0-893b2022571f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:27.693671 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-cc1646c4-409d-4a34-8cc0-893b2022571f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:27.786691 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:57:27.803395 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1646c4-409d-4a34-8cc0-893b2022571f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:57:27.804867 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1646c4-409d-4a34-8cc0-893b2022571f service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:57:27.823619 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 672/1341] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:57:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:57:30.997931 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f8f18-22fe-4f91-8967-bd32b18ff268 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:30.998474 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f8f18-22fe-4f91-8967-bd32b18ff268 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:30.998780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f8f18-22fe-4f91-8967-bd32b18ff268 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:31.001123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 670/1342] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:57:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:57:31.008045 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f476c5-17ec-4603-b6c0-e5a6651a2a10 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:31.008803 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f476c5-17ec-4603-b6c0-e5a6651a2a10 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:31.008803 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f476c5-17ec-4603-b6c0-e5a6651a2a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:31.084476 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-c8f476c5-17ec-4603-b6c0-e5a6651a2a10 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:57:31.085688 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c8f476c5-17ec-4603-b6c0-e5a6651a2a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:31.180106 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 673/1343] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:57:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:57:31.478368 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7a698-0a40-46fe-990e-1a5a516ce755 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:31.478718 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7a698-0a40-46fe-990e-1a5a516ce755 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:31.479052 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7a698-0a40-46fe-990e-1a5a516ce755 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:31.480572 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 671/1344] 10.222.0.48 () {54 vars in 965 bytes} [Fri Sep 9 20:57:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:57:31.486650 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03f13e-06b5-40cd-a84f-60c27e2532f4 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:31.487003 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03f13e-06b5-40cd-a84f-60c27e2532f4 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:31.487279 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03f13e-06b5-40cd-a84f-60c27e2532f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:31.555481 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:57:31.564747 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-8c03f13e-06b5-40cd-a84f-60c27e2532f4 None None] MFA Rules not processed for user `8f5dfac06ac9492a86503ba133c638a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:57:31.565947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8c03f13e-06b5-40cd-a84f-60c27e2532f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:31.643719 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 674/1345] 10.222.0.48 () {58 vars in 1062 bytes} [Fri Sep 9 20:57:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:57:31.652715 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7a698-0a40-46fe-990e-1a5a516ce755 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:57:31.654294 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b0a7a698-0a40-46fe-990e-1a5a516ce755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:31.776581 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7a698-0a40-46fe-990e-1a5a516ce755 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:57:31.856621 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1376a-d5bc-487e-a7ce-80078c756240 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:57:31.857237 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1376a-d5bc-487e-a7ce-80078c756240 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:31.857513 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1376a-d5bc-487e-a7ce-80078c756240 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:31.857782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1376a-d5bc-487e-a7ce-80078c756240 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:31.859466 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e6a1376a-d5bc-487e-a7ce-80078c756240 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:31.905214 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:57:31.978020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a1376a-d5bc-487e-a7ce-80078c756240 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:57:31.979479 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a1376a-d5bc-487e-a7ce-80078c756240 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:57:32.002030 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 672/1346] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:57:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:57:37.033845 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a3c6e-281f-4690-b588-be0914db2e7d None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:37.033845 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a3c6e-281f-4690-b588-be0914db2e7d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:37.033845 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a3c6e-281f-4690-b588-be0914db2e7d None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:37.033845 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 675/1347] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:57:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:57:37.034567 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824b98b-6746-4ae0-bc18-6062de510500 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:37.034567 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824b98b-6746-4ae0-bc18-6062de510500 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:37.034567 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824b98b-6746-4ae0-bc18-6062de510500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:37.103202 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f824b98b-6746-4ae0-bc18-6062de510500 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:57:37.104511 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f824b98b-6746-4ae0-bc18-6062de510500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:37.202367 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:57:37.204642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 673/1348] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:57:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:57:37.493548 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a3c6e-281f-4690-b588-be0914db2e7d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:57:37.494995 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-061a3c6e-281f-4690-b588-be0914db2e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:37.604304 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a3c6e-281f-4690-b588-be0914db2e7d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:57:37.653786 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:57:37.679862 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af69be-2d71-4ccf-a3ad-aef269cf9b77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:57:37.680551 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af69be-2d71-4ccf-a3ad-aef269cf9b77 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:37.680835 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af69be-2d71-4ccf-a3ad-aef269cf9b77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:37.681104 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af69be-2d71-4ccf-a3ad-aef269cf9b77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:37.682709 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-39af69be-2d71-4ccf-a3ad-aef269cf9b77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:37.796234 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39af69be-2d71-4ccf-a3ad-aef269cf9b77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:57:37.797767 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39af69be-2d71-4ccf-a3ad-aef269cf9b77 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:57:37.816601 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 676/1349] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:57:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:57:42.779669 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630a9e9-e13e-438a-981f-a7d021a7f3e7 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:42.780786 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630a9e9-e13e-438a-981f-a7d021a7f3e7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:42.781279 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630a9e9-e13e-438a-981f-a7d021a7f3e7 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:42.782835 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 674/1350] 10.222.0.48 () {54 vars in 921 bytes} [Fri Sep 9 20:57:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:57:42.791853 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e6e90-3c60-4f36-b0ae-add172d0823a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:42.791853 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e6e90-3c60-4f36-b0ae-add172d0823a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:42.791853 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e6e90-3c60-4f36-b0ae-add172d0823a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:42.880968 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-cf1e6e90-3c60-4f36-b0ae-add172d0823a None None] MFA Rules not processed for user `2a4f6de2065d4fbea3fa141da760f708`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:57:42.882210 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-cf1e6e90-3c60-4f36-b0ae-add172d0823a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:42.931345 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:57:43.055063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 677/1351] 10.222.0.48 () {58 vars in 1018 bytes} [Fri Sep 9 20:57:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:57:47.131124 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916a33c-e9f8-44a9-b219-fe90f373b4b4 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:47.131718 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916a33c-e9f8-44a9-b219-fe90f373b4b4 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:47.131976 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916a33c-e9f8-44a9-b219-fe90f373b4b4 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:47.133459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 675/1352] 10.222.0.48 () {54 vars in 921 bytes} [Fri Sep 9 20:57:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:57:47.138959 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c66469-e9e1-49cd-bd6a-f3b5669f83d7 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:47.139397 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c66469-e9e1-49cd-bd6a-f3b5669f83d7 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:47.139507 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c66469-e9e1-49cd-bd6a-f3b5669f83d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:47.221141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-91c66469-e9e1-49cd-bd6a-f3b5669f83d7 None None] MFA Rules not processed for user `2a4f6de2065d4fbea3fa141da760f708`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:57:47.222641 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-91c66469-e9e1-49cd-bd6a-f3b5669f83d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:47.301818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:57:47.346380 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 678/1353] 10.222.0.48 () {58 vars in 1018 bytes} [Fri Sep 9 20:57:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:57:48.933116 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4678f4c-6a6a-4020-aa63-d95e53125f2e None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:48.933700 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4678f4c-6a6a-4020-aa63-d95e53125f2e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:48.933803 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4678f4c-6a6a-4020-aa63-d95e53125f2e None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:48.935083 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 676/1354] 10.222.0.48 () {54 vars in 921 bytes} [Fri Sep 9 20:57:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:57:48.940491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a192a0-de51-42d7-bfa6-6f737e79c3f5 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:48.940978 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a192a0-de51-42d7-bfa6-6f737e79c3f5 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:48.941197 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a192a0-de51-42d7-bfa6-6f737e79c3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:49.020050 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-57a192a0-de51-42d7-bfa6-6f737e79c3f5 None None] MFA Rules not processed for user `2a4f6de2065d4fbea3fa141da760f708`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:57:49.021289 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-57a192a0-de51-42d7-bfa6-6f737e79c3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:49.133330 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 679/1355] 10.222.0.48 () {58 vars in 1018 bytes} [Fri Sep 9 20:57:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:57:52.544180 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2788bd-cfde-48c5-a3b1-7ce12d895bc9 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:52.544180 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2788bd-cfde-48c5-a3b1-7ce12d895bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:52.544180 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2788bd-cfde-48c5-a3b1-7ce12d895bc9 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:52.545597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 677/1356] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:57:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:57:52.551392 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7dc362-3b03-44cd-a146-8879e862f3a4 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:52.551855 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7dc362-3b03-44cd-a146-8879e862f3a4 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:52.551939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7dc362-3b03-44cd-a146-8879e862f3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:52.592499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:57:52.625882 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-cc7dc362-3b03-44cd-a146-8879e862f3a4 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:57:52.627869 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-cc7dc362-3b03-44cd-a146-8879e862f3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:52.734537 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 680/1357] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:57:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:57:53.040786 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536c777-e07d-4a71-9350-176300190c0c None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:53.041336 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536c777-e07d-4a71-9350-176300190c0c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:53.042069 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536c777-e07d-4a71-9350-176300190c0c None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:53.044434 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 678/1358] 10.222.0.48 () {54 vars in 962 bytes} [Fri Sep 9 20:57:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:57:53.057157 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c0496-59cb-4885-9f12-581c478dd091 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:53.057393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c0496-59cb-4885-9f12-581c478dd091 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:53.057687 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c0496-59cb-4885-9f12-581c478dd091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:53.140769 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-595c0496-59cb-4885-9f12-581c478dd091 None None] MFA Rules not processed for user `613b845a9b4d432daf8d478fc3970708`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:57:53.142012 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-595c0496-59cb-4885-9f12-581c478dd091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:53.192716 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:57:53.264772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 681/1359] 10.222.0.48 () {58 vars in 1059 bytes} [Fri Sep 9 20:57:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:57:53.272585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3536c777-e07d-4a71-9350-176300190c0c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:57:53.274252 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3536c777-e07d-4a71-9350-176300190c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:53.428012 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3536c777-e07d-4a71-9350-176300190c0c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:57:53.507745 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:57:53.549130 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7b6ea-6639-4961-80af-dbfa012b2c2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:57:53.549719 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7b6ea-6639-4961-80af-dbfa012b2c2c service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:53.549973 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7b6ea-6639-4961-80af-dbfa012b2c2c service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:53.550421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7b6ea-6639-4961-80af-dbfa012b2c2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:53.551859 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-41a7b6ea-6639-4961-80af-dbfa012b2c2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:53.663365 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7b6ea-6639-4961-80af-dbfa012b2c2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:57:53.665905 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7b6ea-6639-4961-80af-dbfa012b2c2c service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:57:53.690386 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 679/1360] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:57:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:57:56.105325 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220146b-7110-4672-9e87-3b96b1a63922 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:56.105758 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220146b-7110-4672-9e87-3b96b1a63922 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:56.105946 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220146b-7110-4672-9e87-3b96b1a63922 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:56.108512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 682/1361] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:57:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:57:56.113682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05965ee-f0f8-44bb-9a6b-24efb3489507 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:56.114151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05965ee-f0f8-44bb-9a6b-24efb3489507 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:56.115017 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05965ee-f0f8-44bb-9a6b-24efb3489507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:56.197376 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a05965ee-f0f8-44bb-9a6b-24efb3489507 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:57:56.199111 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a05965ee-f0f8-44bb-9a6b-24efb3489507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:56.239517 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:57:56.302117 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 680/1362] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:57:56.592355 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220146b-7110-4672-9e87-3b96b1a63922 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:57:56.593874 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2220146b-7110-4672-9e87-3b96b1a63922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:56.783604 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220146b-7110-4672-9e87-3b96b1a63922 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:57:56.794580 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:57:56.920574 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104d9ca-2690-4476-adfc-fdd565b196e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:57:56.921185 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104d9ca-2690-4476-adfc-fdd565b196e4 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:56.921464 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104d9ca-2690-4476-adfc-fdd565b196e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:56.921732 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104d9ca-2690-4476-adfc-fdd565b196e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:56.923524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2104d9ca-2690-4476-adfc-fdd565b196e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:57.019873 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104d9ca-2690-4476-adfc-fdd565b196e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:57:57.021530 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104d9ca-2690-4476-adfc-fdd565b196e4 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:57:57.044571 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 683/1363] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:57:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:57:59.629908 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235dc571-7538-45c3-9a1b-55d6adcd3f4a None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:59.630475 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235dc571-7538-45c3-9a1b-55d6adcd3f4a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:59.630681 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235dc571-7538-45c3-9a1b-55d6adcd3f4a None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:59.632289 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 681/1364] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:57:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:57:59.638207 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f14152-5dea-43fc-ba98-c6c21fa55b9c None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:57:59.638637 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f14152-5dea-43fc-ba98-c6c21fa55b9c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:57:59.638941 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f14152-5dea-43fc-ba98-c6c21fa55b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:57:59.655682 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:57:59.710293 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-b9f14152-5dea-43fc-ba98-c6c21fa55b9c None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:57:59.711657 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b9f14152-5dea-43fc-ba98-c6c21fa55b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:57:59.813960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 684/1365] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:58:00.109063 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf8ccd-0161-414b-91a4-b97084a0452a None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:00.109356 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf8ccd-0161-414b-91a4-b97084a0452a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:00.109630 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf8ccd-0161-414b-91a4-b97084a0452a None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:00.111099 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 682/1366] 10.222.0.48 () {54 vars in 962 bytes} [Fri Sep 9 20:58:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:58:00.116808 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75cb6c-e163-46f5-9b96-b570080393c2 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:00.129338 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75cb6c-e163-46f5-9b96-b570080393c2 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:00.129338 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75cb6c-e163-46f5-9b96-b570080393c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:00.196451 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-fe75cb6c-e163-46f5-9b96-b570080393c2 None None] MFA Rules not processed for user `613b845a9b4d432daf8d478fc3970708`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:58:00.197666 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-fe75cb6c-e163-46f5-9b96-b570080393c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:00.233396 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:00.317149 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 685/1367] 10.222.0.48 () {58 vars in 1059 bytes} [Fri Sep 9 20:58:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:58:00.325823 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf8ccd-0161-414b-91a4-b97084a0452a None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:58:00.327317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-febf8ccd-0161-414b-91a4-b97084a0452a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:00.484015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf8ccd-0161-414b-91a4-b97084a0452a None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:58:00.507196 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:00.604800 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4042a43-9502-4387-a243-5682bebf92ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:58:00.605329 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4042a43-9502-4387-a243-5682bebf92ba service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:00.605598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4042a43-9502-4387-a243-5682bebf92ba service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:00.605862 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4042a43-9502-4387-a243-5682bebf92ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:00.607451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d4042a43-9502-4387-a243-5682bebf92ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:00.708103 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4042a43-9502-4387-a243-5682bebf92ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:58:00.709704 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4042a43-9502-4387-a243-5682bebf92ba service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:58:00.735142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 683/1368] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:58:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:58:00.750065 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee233c7-24d8-4a04-9cd0-a68d7772dfb7 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:00.751136 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee233c7-24d8-4a04-9cd0-a68d7772dfb7 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:00.751474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee233c7-24d8-4a04-9cd0-a68d7772dfb7 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:00.753289 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 686/1369] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:58:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 20:58:00.759741 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267a9a6-0b8d-4b02-b6f0-981dd7800b90 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:00.760661 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267a9a6-0b8d-4b02-b6f0-981dd7800b90 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:00.761334 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267a9a6-0b8d-4b02-b6f0-981dd7800b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:00.796194 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:00.831069 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a267a9a6-0b8d-4b02-b6f0-981dd7800b90 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:58:00.832363 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a267a9a6-0b8d-4b02-b6f0-981dd7800b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:00.949499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 684/1370] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:58:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:58:01.283956 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9ae3a-14eb-41ef-b8c0-503d57f7bd6e None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:01.283956 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9ae3a-14eb-41ef-b8c0-503d57f7bd6e None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:01.283956 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9ae3a-14eb-41ef-b8c0-503d57f7bd6e None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:01.286846 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 687/1371] 10.222.0.48 () {54 vars in 964 bytes} [Fri Sep 9 20:58:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:58:01.291264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b34189-c556-4317-8a5f-987c97b26ed8 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:01.291658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b34189-c556-4317-8a5f-987c97b26ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:01.292003 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b34189-c556-4317-8a5f-987c97b26ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:01.370855 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-00b34189-c556-4317-8a5f-987c97b26ed8 None None] MFA Rules not processed for user `cf91131ee5bb481bbb47097cad0c003a`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:58:01.372167 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-00b34189-c556-4317-8a5f-987c97b26ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:01.424210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:01.456185 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 685/1372] 10.222.0.48 () {58 vars in 1061 bytes} [Fri Sep 9 20:58:01 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:58:01.465227 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9ae3a-14eb-41ef-b8c0-503d57f7bd6e None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:58:01.468134 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3dd9ae3a-14eb-41ef-b8c0-503d57f7bd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:01.578730 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9ae3a-14eb-41ef-b8c0-503d57f7bd6e None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:58:01.622217 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:01.657946 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0fabfa-6662-4bbd-85d2-159f650ca1c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:58:01.658721 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fabfa-6662-4bbd-85d2-159f650ca1c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:01.659095 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fabfa-6662-4bbd-85d2-159f650ca1c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:01.659459 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fabfa-6662-4bbd-85d2-159f650ca1c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:01.661185 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0d0fabfa-6662-4bbd-85d2-159f650ca1c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:01.788821 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0fabfa-6662-4bbd-85d2-159f650ca1c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:58:01.790260 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0fabfa-6662-4bbd-85d2-159f650ca1c1 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:58:01.809256 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 688/1373] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:58:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:58:03.185585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b7154-9880-4a58-83cb-b15e9be5c500 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:03.186124 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b7154-9880-4a58-83cb-b15e9be5c500 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:03.186124 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b7154-9880-4a58-83cb-b15e9be5c500 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:03.187658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 686/1374] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:58:03.193085 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f3294-3f83-4977-bfc2-c923df1c044c None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:03.193511 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f3294-3f83-4977-bfc2-c923df1c044c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:03.193627 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f3294-3f83-4977-bfc2-c923df1c044c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:03.257843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-588f3294-3f83-4977-bfc2-c923df1c044c None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:58:03.259072 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-588f3294-3f83-4977-bfc2-c923df1c044c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:03.280819 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:03.356132 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 689/1375] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:58:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:58:03.649442 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b7154-9880-4a58-83cb-b15e9be5c500 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:58:03.651010 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fc2b7154-9880-4a58-83cb-b15e9be5c500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:03.803364 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b7154-9880-4a58-83cb-b15e9be5c500 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:58:03.856212 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:03.922637 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e26f4-d774-4a56-9cf1-85f4556df570 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:58:03.923247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e26f4-d774-4a56-9cf1-85f4556df570 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:03.923522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e26f4-d774-4a56-9cf1-85f4556df570 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:03.923786 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e26f4-d774-4a56-9cf1-85f4556df570 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:03.925403 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-dd9e26f4-d774-4a56-9cf1-85f4556df570 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:04.024660 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9e26f4-d774-4a56-9cf1-85f4556df570 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:58:04.026831 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9e26f4-d774-4a56-9cf1-85f4556df570 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:58:04.045698 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 687/1376] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:58:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:58:06.412130 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4425c7-31d2-49d4-abd4-c94c4671e65e None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:06.412681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4425c7-31d2-49d4-abd4-c94c4671e65e None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:06.412681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4425c7-31d2-49d4-abd4-c94c4671e65e None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:06.414153 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 690/1377] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:58:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:58:06.419235 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3fff6-b8bf-4fe1-b8dc-f29114a599ea None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:06.419667 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3fff6-b8bf-4fe1-b8dc-f29114a599ea None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:06.419834 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3fff6-b8bf-4fe1-b8dc-f29114a599ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:06.486581 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-13d3fff6-b8bf-4fe1-b8dc-f29114a599ea None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:58:06.487789 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-13d3fff6-b8bf-4fe1-b8dc-f29114a599ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:06.495482 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:06.587028 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 688/1378] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:58:06.874489 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4425c7-31d2-49d4-abd4-c94c4671e65e None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:58:06.876268 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ad4425c7-31d2-49d4-abd4-c94c4671e65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:07.036942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4425c7-31d2-49d4-abd4-c94c4671e65e None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:58:07.040937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:07.158063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d82e70-79dd-436c-a26b-8e71ba87cf37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:58:07.158675 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d82e70-79dd-436c-a26b-8e71ba87cf37 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:07.158953 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d82e70-79dd-436c-a26b-8e71ba87cf37 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:07.159222 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d82e70-79dd-436c-a26b-8e71ba87cf37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:07.160841 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e0d82e70-79dd-436c-a26b-8e71ba87cf37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:07.254082 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d82e70-79dd-436c-a26b-8e71ba87cf37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:58:07.255460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d82e70-79dd-436c-a26b-8e71ba87cf37 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:58:07.274812 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 691/1379] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:58:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:58:09.981554 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219e4e6-ef2b-4920-983b-796f143b13b5 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:09.982102 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219e4e6-ef2b-4920-983b-796f143b13b5 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:09.982102 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219e4e6-ef2b-4920-983b-796f143b13b5 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:09.983563 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 689/1380] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:58:09.989462 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1075a-5e36-4965-a54c-001c26f4a251 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:09.989938 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1075a-5e36-4965-a54c-001c26f4a251 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:09.990070 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1075a-5e36-4965-a54c-001c26f4a251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:10.002918 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:10.064298 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-f4c1075a-5e36-4965-a54c-001c26f4a251 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:58:10.065588 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f4c1075a-5e36-4965-a54c-001c26f4a251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:10.171387 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 692/1381] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:58:10.563696 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d219e4e6-ef2b-4920-983b-796f143b13b5 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:58:10.565352 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d219e4e6-ef2b-4920-983b-796f143b13b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:10.741696 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:10.757844 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d219e4e6-ef2b-4920-983b-796f143b13b5 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:58:10.916408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041fa91c-ee3c-4970-a7d0-13eeb75a49d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:58:10.917048 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041fa91c-ee3c-4970-a7d0-13eeb75a49d0 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:10.917327 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041fa91c-ee3c-4970-a7d0-13eeb75a49d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:10.917607 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041fa91c-ee3c-4970-a7d0-13eeb75a49d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:10.919258 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-041fa91c-ee3c-4970-a7d0-13eeb75a49d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:11.048142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041fa91c-ee3c-4970-a7d0-13eeb75a49d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:58:11.049828 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041fa91c-ee3c-4970-a7d0-13eeb75a49d0 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:58:11.060242 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:11.069515 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 690/1382] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:58:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:58:13.626673 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6f3b9-24a6-487d-8526-64e11e20b8d3 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:13.627235 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6f3b9-24a6-487d-8526-64e11e20b8d3 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:13.627235 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6f3b9-24a6-487d-8526-64e11e20b8d3 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:13.628648 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 693/1383] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:58:13.634309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813712f-05ff-4b66-b335-bed8bb0f279b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:13.634831 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813712f-05ff-4b66-b335-bed8bb0f279b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:13.634988 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813712f-05ff-4b66-b335-bed8bb0f279b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:13.707290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-7813712f-05ff-4b66-b335-bed8bb0f279b None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:58:13.708815 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7813712f-05ff-4b66-b335-bed8bb0f279b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:13.813181 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 691/1384] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:58:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:58:14.099634 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae6f3b9-24a6-487d-8526-64e11e20b8d3 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:58:14.101321 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-fae6f3b9-24a6-487d-8526-64e11e20b8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:14.189077 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:14.265141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae6f3b9-24a6-487d-8526-64e11e20b8d3 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:58:14.408505 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b07d9-d3d1-44e5-9c2b-645bb1aed275 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:58:14.409644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b07d9-d3d1-44e5-9c2b-645bb1aed275 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:14.409923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b07d9-d3d1-44e5-9c2b-645bb1aed275 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:14.410332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b07d9-d3d1-44e5-9c2b-645bb1aed275 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:14.412488 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-830b07d9-d3d1-44e5-9c2b-645bb1aed275 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:14.475451 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:14.517649 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830b07d9-d3d1-44e5-9c2b-645bb1aed275 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:58:14.519238 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830b07d9-d3d1-44e5-9c2b-645bb1aed275 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:58:14.538171 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 694/1385] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:58:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:58:16.781568 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4528a-046f-490a-9ddf-92a5165e8a43 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:16.782196 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4528a-046f-490a-9ddf-92a5165e8a43 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:16.782196 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4528a-046f-490a-9ddf-92a5165e8a43 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:16.783527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 692/1386] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:58:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:58:16.788866 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00877e8d-77f9-422f-ba0c-0c1c8cfe2058 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:16.789301 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00877e8d-77f9-422f-ba0c-0c1c8cfe2058 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:16.789547 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00877e8d-77f9-422f-ba0c-0c1c8cfe2058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:16.860507 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-00877e8d-77f9-422f-ba0c-0c1c8cfe2058 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:58:16.861706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-00877e8d-77f9-422f-ba0c-0c1c8cfe2058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:16.960399 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 695/1387] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:58:17.248048 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b4528a-046f-490a-9ddf-92a5165e8a43 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:58:17.249657 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-27b4528a-046f-490a-9ddf-92a5165e8a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:17.328096 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:17.397872 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b4528a-046f-490a-9ddf-92a5165e8a43 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:58:17.516939 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f55dff-731f-4b52-8059-fecb8252bc29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:58:17.517550 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f55dff-731f-4b52-8059-fecb8252bc29 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:17.517828 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f55dff-731f-4b52-8059-fecb8252bc29 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:17.518100 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f55dff-731f-4b52-8059-fecb8252bc29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:17.519776 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-06f55dff-731f-4b52-8059-fecb8252bc29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:17.558120 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:17.609034 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f55dff-731f-4b52-8059-fecb8252bc29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:58:17.610657 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f55dff-731f-4b52-8059-fecb8252bc29 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:58:17.628988 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 693/1388] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:58:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:58:18.248061 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035743a-9ab4-473c-8cf2-c1acad4304c9 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:18.248674 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035743a-9ab4-473c-8cf2-c1acad4304c9 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:18.248674 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035743a-9ab4-473c-8cf2-c1acad4304c9 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:18.249955 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 696/1389] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:58:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:58:18.255213 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737065c-b0a5-406c-80b5-5048d3126409 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:18.255669 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737065c-b0a5-406c-80b5-5048d3126409 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:18.255788 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737065c-b0a5-406c-80b5-5048d3126409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:18.323410 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-7737065c-b0a5-406c-80b5-5048d3126409 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:58:18.324639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7737065c-b0a5-406c-80b5-5048d3126409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:18.423344 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 694/1390] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:58:18.733999 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab87bb8-7284-47cd-a80d-3ad9eadf7e6a None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:18.734297 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab87bb8-7284-47cd-a80d-3ad9eadf7e6a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:18.734815 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab87bb8-7284-47cd-a80d-3ad9eadf7e6a None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:18.736313 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 697/1391] 10.222.0.48 () {54 vars in 966 bytes} [Fri Sep 9 20:58:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:58:18.742131 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be96ae-d0ff-44f2-8fda-4495a2aa5b6d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:18.742494 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be96ae-d0ff-44f2-8fda-4495a2aa5b6d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:18.742845 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be96ae-d0ff-44f2-8fda-4495a2aa5b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:18.759339 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:18.824231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-68be96ae-d0ff-44f2-8fda-4495a2aa5b6d None None] MFA Rules not processed for user `7832e97d7e97439e821268d05816ae3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:58:18.825793 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-68be96ae-d0ff-44f2-8fda-4495a2aa5b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:18.906098 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 695/1392] 10.222.0.48 () {58 vars in 1063 bytes} [Fri Sep 9 20:58:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:58:18.915270 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab87bb8-7284-47cd-a80d-3ad9eadf7e6a None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:58:18.916856 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-9ab87bb8-7284-47cd-a80d-3ad9eadf7e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:18.949424 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:19.029504 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab87bb8-7284-47cd-a80d-3ad9eadf7e6a None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:58:19.103400 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fbbdb2-5c97-4a06-95a7-82f20b367ae8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7832e97d7e97439e821268d05816ae3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:58:19.103922 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fbbdb2-5c97-4a06-95a7-82f20b367ae8 service glance] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:19.104239 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fbbdb2-5c97-4a06-95a7-82f20b367ae8 service glance] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:19.104509 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fbbdb2-5c97-4a06-95a7-82f20b367ae8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:19.106075 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c5fbbdb2-5c97-4a06-95a7-82f20b367ae8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:19.184012 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:19.215846 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fbbdb2-5c97-4a06-95a7-82f20b367ae8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:58:19.217318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fbbdb2-5c97-4a06-95a7-82f20b367ae8 service glance] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:58:19.235512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 698/1393] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:58:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:58:19.962516 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d33c6f-ff8a-467d-a45d-618467f4df7e None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:19.963633 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d33c6f-ff8a-467d-a45d-618467f4df7e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:19.963633 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d33c6f-ff8a-467d-a45d-618467f4df7e None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:19.965302 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 696/1394] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:58:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:58:19.971002 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148ec93-10d2-4465-8db8-b1d3acb7c42f None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:19.971462 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148ec93-10d2-4465-8db8-b1d3acb7c42f None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:19.971622 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148ec93-10d2-4465-8db8-b1d3acb7c42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:20.041591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-f148ec93-10d2-4465-8db8-b1d3acb7c42f None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:58:20.042797 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f148ec93-10d2-4465-8db8-b1d3acb7c42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:20.137544 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 699/1395] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:58:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:58:20.153859 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c64b7-c1a1-41a9-aa50-725f316482df None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:20.154150 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c64b7-c1a1-41a9-aa50-725f316482df None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:20.154424 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c64b7-c1a1-41a9-aa50-725f316482df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:20.234624 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5a5c64b7-c1a1-41a9-aa50-725f316482df None None] MFA Rules not processed for user `8c21ea7d3cbb494886e60a4bd4f39145`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:58:20.234624 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5a5c64b7-c1a1-41a9-aa50-725f316482df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:20.276954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:20.314649 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 697/1396] 10.222.0.48 () {58 vars in 1009 bytes} [Fri Sep 9 20:58:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:58:20.361255 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1f119-fdd9-445b-9d4f-d719440ff585 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:20.361547 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1f119-fdd9-445b-9d4f-d719440ff585 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:20.361823 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1f119-fdd9-445b-9d4f-d719440ff585 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:20.363162 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 700/1397] 10.222.0.48 () {54 vars in 943 bytes} [Fri Sep 9 20:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:58:20.370828 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a17398-4f60-42f7-8d7a-58f50d353f45 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:20.371116 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a17398-4f60-42f7-8d7a-58f50d353f45 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:20.371391 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a17398-4f60-42f7-8d7a-58f50d353f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:20.426362 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb1f119-fdd9-445b-9d4f-d719440ff585 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:58:20.428574 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2cb1f119-fdd9-445b-9d4f-d719440ff585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:20.459639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-40a17398-4f60-42f7-8d7a-58f50d353f45 None None] MFA Rules not processed for user `7d28fa5d49d44120bd5eaec2d4e5fdbd`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:58:20.460958 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-40a17398-4f60-42f7-8d7a-58f50d353f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:20.472605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:20.574307 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 698/1398] 10.222.0.48 () {58 vars in 1040 bytes} [Fri Sep 9 20:58:20 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:58:20.581934 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb1f119-fdd9-445b-9d4f-d719440ff585 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:58:20.584112 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a17398-4f60-42f7-8d7a-58f50d353f45 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:58:20.593867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a17398-4f60-42f7-8d7a-58f50d353f45 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:58:20.620023 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:20.701188 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0cd9f-f40c-414c-8351-b19b3a6082e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:58:20.701805 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0cd9f-f40c-414c-8351-b19b3a6082e1 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:20.702085 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0cd9f-f40c-414c-8351-b19b3a6082e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:20.702363 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0cd9f-f40c-414c-8351-b19b3a6082e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:20.704036 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3da0cd9f-f40c-414c-8351-b19b3a6082e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:20.704513 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab90586b-441d-427d-8731-83c6de8847c4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d28fa5d49d44120bd5eaec2d4e5fdbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:58:20.705116 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab90586b-441d-427d-8731-83c6de8847c4 service swift] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:20.705393 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab90586b-441d-427d-8731-83c6de8847c4 service swift] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:20.705662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab90586b-441d-427d-8731-83c6de8847c4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:20.707272 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ab90586b-441d-427d-8731-83c6de8847c4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:20.756755 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:20.793118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab90586b-441d-427d-8731-83c6de8847c4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:58:20.794756 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab90586b-441d-427d-8731-83c6de8847c4 service swift] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:58:20.798407 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0cd9f-f40c-414c-8351-b19b3a6082e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:58:20.799988 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0cd9f-f40c-414c-8351-b19b3a6082e1 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:58:20.802849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 699/1399] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Sep 9 20:58:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 20:58:20.822048 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 701/1400] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:58:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 20:58:23.130433 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a29d2-d66a-4f11-a5e3-c8d50e531c37 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:23.131000 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a29d2-d66a-4f11-a5e3-c8d50e531c37 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:23.131000 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a29d2-d66a-4f11-a5e3-c8d50e531c37 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:23.132710 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 700/1401] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:58:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:58:23.140328 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da887c09-41b7-4db9-9f3d-5deaac33ae7f None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:23.140328 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da887c09-41b7-4db9-9f3d-5deaac33ae7f None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:23.140328 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da887c09-41b7-4db9-9f3d-5deaac33ae7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:23.219582 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-da887c09-41b7-4db9-9f3d-5deaac33ae7f None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:58:23.221568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-da887c09-41b7-4db9-9f3d-5deaac33ae7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:23.300371 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:23.344142 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 702/1402] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:58:23.625620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a29d2-d66a-4f11-a5e3-c8d50e531c37 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:58:23.627141 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9e6a29d2-d66a-4f11-a5e3-c8d50e531c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:23.694815 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:23.786901 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a29d2-d66a-4f11-a5e3-c8d50e531c37 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:58:23.937683 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707feed5-835f-416c-bb80-27a5ffad13cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:58:23.938300 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707feed5-835f-416c-bb80-27a5ffad13cc service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:23.938585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707feed5-835f-416c-bb80-27a5ffad13cc service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:23.938851 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707feed5-835f-416c-bb80-27a5ffad13cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:23.940547 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-707feed5-835f-416c-bb80-27a5ffad13cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:23.961174 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:24.048845 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707feed5-835f-416c-bb80-27a5ffad13cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:58:24.050459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707feed5-835f-416c-bb80-27a5ffad13cc service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:58:24.072281 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 701/1403] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:58:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:58:26.466851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff02a34-13a8-4030-a1c3-bdc8c7fc0cb6 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:26.467449 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff02a34-13a8-4030-a1c3-bdc8c7fc0cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:26.467449 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff02a34-13a8-4030-a1c3-bdc8c7fc0cb6 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:26.468893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 703/1404] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:58:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:58:26.478232 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a135a2-28fc-4447-8f88-462619cdc16c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:26.478814 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a135a2-28fc-4447-8f88-462619cdc16c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:26.478900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a135a2-28fc-4447-8f88-462619cdc16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:26.548290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e1a135a2-28fc-4447-8f88-462619cdc16c None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:58:26.549517 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e1a135a2-28fc-4447-8f88-462619cdc16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:26.653476 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:26.659226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 702/1405] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:58:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:58:26.965791 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff02a34-13a8-4030-a1c3-bdc8c7fc0cb6 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:58:26.967329 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-7ff02a34-13a8-4030-a1c3-bdc8c7fc0cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:27.136876 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff02a34-13a8-4030-a1c3-bdc8c7fc0cb6 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:58:27.197391 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:27.266919 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a1f5e-accb-469e-9bc9-2b28e7ba61d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:58:27.267928 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a1f5e-accb-469e-9bc9-2b28e7ba61d0 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:27.268352 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a1f5e-accb-469e-9bc9-2b28e7ba61d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:27.268734 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a1f5e-accb-469e-9bc9-2b28e7ba61d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:27.270435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-239a1f5e-accb-469e-9bc9-2b28e7ba61d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:27.373976 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a1f5e-accb-469e-9bc9-2b28e7ba61d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:58:27.374219 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a1f5e-accb-469e-9bc9-2b28e7ba61d0 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:58:27.393298 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 704/1406] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:58:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:58:29.710862 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af721e-64fa-4aa1-bb77-143e2aedbe56 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:29.711418 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af721e-64fa-4aa1-bb77-143e2aedbe56 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:29.711418 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af721e-64fa-4aa1-bb77-143e2aedbe56 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:29.713603 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 703/1407] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:58:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:58:29.718729 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0a4e3-02b7-44d6-8397-19d71276e015 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:29.719158 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0a4e3-02b7-44d6-8397-19d71276e015 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:29.719259 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0a4e3-02b7-44d6-8397-19d71276e015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:29.787981 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-47a0a4e3-02b7-44d6-8397-19d71276e015 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:58:29.789483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-47a0a4e3-02b7-44d6-8397-19d71276e015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:29.803351 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:29.894522 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 705/1408] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:58:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:58:30.187971 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af721e-64fa-4aa1-bb77-143e2aedbe56 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:58:30.189604 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c9af721e-64fa-4aa1-bb77-143e2aedbe56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:30.359501 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af721e-64fa-4aa1-bb77-143e2aedbe56 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:58:30.437680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:30.491368 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567d062-1d13-4152-91a9-63d2fa8714d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:58:30.492151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567d062-1d13-4152-91a9-63d2fa8714d9 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:30.492441 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567d062-1d13-4152-91a9-63d2fa8714d9 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:30.492702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567d062-1d13-4152-91a9-63d2fa8714d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:30.494908 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5567d062-1d13-4152-91a9-63d2fa8714d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:30.594906 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5567d062-1d13-4152-91a9-63d2fa8714d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:58:30.597105 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5567d062-1d13-4152-91a9-63d2fa8714d9 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:58:30.620370 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 704/1409] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:58:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:58:39.894951 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=38 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:40.616934 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=64 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:44.994918 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db4157-4a19-4179-8e16-d4c53ea89d46 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:44.995489 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db4157-4a19-4179-8e16-d4c53ea89d46 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:44.995489 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db4157-4a19-4179-8e16-d4c53ea89d46 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:44.996818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 706/1410] 10.222.0.48 () {54 vars in 918 bytes} [Fri Sep 9 20:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:58:45.001942 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9574fa7-6625-4d8c-abe2-9550dab3bf41 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:45.002355 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9574fa7-6625-4d8c-abe2-9550dab3bf41 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:45.002477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9574fa7-6625-4d8c-abe2-9550dab3bf41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:45.077545 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a9574fa7-6625-4d8c-abe2-9550dab3bf41 None None] MFA Rules not processed for user `2a4f6de2065d4fbea3fa141da760f708`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:58:45.078736 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a9574fa7-6625-4d8c-abe2-9550dab3bf41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:45.181962 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 705/1411] 10.222.0.48 () {58 vars in 1015 bytes} [Fri Sep 9 20:58:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:58:45.196603 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff2380-59f0-4a77-907b-850df4369ac2 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:45.196895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff2380-59f0-4a77-907b-850df4369ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:45.197172 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff2380-59f0-4a77-907b-850df4369ac2 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:45.198486 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 707/1412] 10.222.0.48 () {54 vars in 918 bytes} [Fri Sep 9 20:58:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:58:45.203113 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2fff7-bf1e-4e52-86db-c8e100d57aba None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:45.203395 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2fff7-bf1e-4e52-86db-c8e100d57aba None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:45.203667 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2fff7-bf1e-4e52-86db-c8e100d57aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:45.274567 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:45.278004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d0c2fff7-bf1e-4e52-86db-c8e100d57aba None None] MFA Rules not processed for user `613b845a9b4d432daf8d478fc3970708`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:58:45.279193 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d0c2fff7-bf1e-4e52-86db-c8e100d57aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:45.392090 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 706/1413] 10.222.0.48 () {58 vars in 1015 bytes} [Fri Sep 9 20:58:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:58:47.952553 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57502a-7d85-4fd2-ba7f-9aa3a07ada4c None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:47.953141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57502a-7d85-4fd2-ba7f-9aa3a07ada4c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:47.953141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57502a-7d85-4fd2-ba7f-9aa3a07ada4c None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:47.954568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 20:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:58:47.959566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c37dae-db82-4a74-a338-32d0e4ec1243 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:47.960007 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c37dae-db82-4a74-a338-32d0e4ec1243 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:47.960156 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c37dae-db82-4a74-a338-32d0e4ec1243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:48.034459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-29c37dae-db82-4a74-a338-32d0e4ec1243 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:58:48.035664 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-29c37dae-db82-4a74-a338-32d0e4ec1243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:48.089912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:48.130478 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 20:58:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:58:48.426905 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57502a-7d85-4fd2-ba7f-9aa3a07ada4c None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:58:48.428473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-bc57502a-7d85-4fd2-ba7f-9aa3a07ada4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:48.579058 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57502a-7d85-4fd2-ba7f-9aa3a07ada4c None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:58:48.652431 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:48.692271 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465e4f5-34d4-40e5-8701-d4e431bc2941 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:58:48.692874 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465e4f5-34d4-40e5-8701-d4e431bc2941 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:48.693400 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465e4f5-34d4-40e5-8701-d4e431bc2941 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:48.693676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465e4f5-34d4-40e5-8701-d4e431bc2941 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:48.695268 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c465e4f5-34d4-40e5-8701-d4e431bc2941 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:48.782844 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c465e4f5-34d4-40e5-8701-d4e431bc2941 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:58:48.784457 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c465e4f5-34d4-40e5-8701-d4e431bc2941 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:58:48.802373 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 709/1416] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:58:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:58:49.940654 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eed6bf-cd34-43ca-8b10-ac6685fdd9b0 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:49.941210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eed6bf-cd34-43ca-8b10-ac6685fdd9b0 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:49.941210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eed6bf-cd34-43ca-8b10-ac6685fdd9b0 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:49.942555 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 20:58:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:58:49.949535 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab406e22-8910-4e8d-ba28-47b5500f10b7 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:49.949993 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab406e22-8910-4e8d-ba28-47b5500f10b7 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:49.950101 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab406e22-8910-4e8d-ba28-47b5500f10b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:50.024269 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-ab406e22-8910-4e8d-ba28-47b5500f10b7 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:58:50.025485 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ab406e22-8910-4e8d-ba28-47b5500f10b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:50.052254 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:50.121706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 20:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:58:50.271509 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eed6bf-cd34-43ca-8b10-ac6685fdd9b0 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:58:50.272965 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b5eed6bf-cd34-43ca-8b10-ac6685fdd9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:50.414794 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eed6bf-cd34-43ca-8b10-ac6685fdd9b0 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:58:50.455587 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:50.538045 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c62f56-108c-40af-b18b-2cf5dba40f15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:58:50.538569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c62f56-108c-40af-b18b-2cf5dba40f15 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:50.538850 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c62f56-108c-40af-b18b-2cf5dba40f15 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:50.539122 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c62f56-108c-40af-b18b-2cf5dba40f15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:50.540748 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-20c62f56-108c-40af-b18b-2cf5dba40f15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:50.629227 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c62f56-108c-40af-b18b-2cf5dba40f15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:58:50.630826 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c62f56-108c-40af-b18b-2cf5dba40f15 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:58:50.648543 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 709/1419] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:58:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:58:51.778630 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c8bd4-372b-4116-b948-5087a5682b30 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:51.779169 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c8bd4-372b-4116-b948-5087a5682b30 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:51.779169 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c8bd4-372b-4116-b948-5087a5682b30 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:51.780557 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 20:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:58:51.786891 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9978f-7232-4152-9482-c8329c7bafc9 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:51.787296 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9978f-7232-4152-9482-c8329c7bafc9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:51.787396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9978f-7232-4152-9482-c8329c7bafc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:51.841750 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:51.863332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-0cd9978f-7232-4152-9482-c8329c7bafc9 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:58:51.864572 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0cd9978f-7232-4152-9482-c8329c7bafc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:51.963185 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 20:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:58:52.117366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c8bd4-372b-4116-b948-5087a5682b30 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:58:52.118857 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f22c8bd4-372b-4116-b948-5087a5682b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:52.265978 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c8bd4-372b-4116-b948-5087a5682b30 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:58:52.278366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:52.377150 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91dfc6-bad0-4d18-827d-433c1e39d963 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:58:52.377681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91dfc6-bad0-4d18-827d-433c1e39d963 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:52.377958 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91dfc6-bad0-4d18-827d-433c1e39d963 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:52.378227 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91dfc6-bad0-4d18-827d-433c1e39d963 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:52.379811 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ce91dfc6-bad0-4d18-827d-433c1e39d963 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:52.467413 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce91dfc6-bad0-4d18-827d-433c1e39d963 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:58:52.469023 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce91dfc6-bad0-4d18-827d-433c1e39d963 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:58:52.492389 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 712/1422] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:58:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:58:53.602665 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b07dc-3b0c-4ea4-a915-af3b14905633 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:53.603119 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b07dc-3b0c-4ea4-a915-af3b14905633 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:53.603223 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b07dc-3b0c-4ea4-a915-af3b14905633 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:53.604662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 20:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:58:53.611426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d3099-212e-4d86-b63d-c44a4423be84 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:53.612326 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d3099-212e-4d86-b63d-c44a4423be84 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:53.612618 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d3099-212e-4d86-b63d-c44a4423be84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:53.634109 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:53.690171 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-8a1d3099-212e-4d86-b63d-c44a4423be84 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:58:53.691412 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8a1d3099-212e-4d86-b63d-c44a4423be84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:53.788217 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 20:58:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:58:53.938810 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b07dc-3b0c-4ea4-a915-af3b14905633 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:58:53.940423 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1c3b07dc-3b0c-4ea4-a915-af3b14905633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:54.051349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:54.086660 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b07dc-3b0c-4ea4-a915-af3b14905633 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:58:54.197329 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec592d-080e-496b-93e4-43c239374378 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:58:54.198617 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec592d-080e-496b-93e4-43c239374378 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:54.198897 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec592d-080e-496b-93e4-43c239374378 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:54.199190 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec592d-080e-496b-93e4-43c239374378 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:54.200800 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-23ec592d-080e-496b-93e4-43c239374378 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:54.274952 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:54.287788 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ec592d-080e-496b-93e4-43c239374378 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:58:54.289404 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ec592d-080e-496b-93e4-43c239374378 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:58:54.307086 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 712/1425] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:58:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:58:55.427362 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a722c67-130e-41df-bedd-d678c548dbc1 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:55.427925 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a722c67-130e-41df-bedd-d678c548dbc1 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:55.427925 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a722c67-130e-41df-bedd-d678c548dbc1 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:55.429256 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 20:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:58:55.435388 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37591e0-279c-440a-b8cb-d501e2ad636a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:55.435833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37591e0-279c-440a-b8cb-d501e2ad636a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:55.435943 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37591e0-279c-440a-b8cb-d501e2ad636a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:55.510410 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c37591e0-279c-440a-b8cb-d501e2ad636a None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:58:55.512156 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c37591e0-279c-440a-b8cb-d501e2ad636a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:55.607019 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 20:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:58:55.756723 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a722c67-130e-41df-bedd-d678c548dbc1 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:58:55.758219 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5a722c67-130e-41df-bedd-d678c548dbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:55.841757 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:55.902286 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a722c67-130e-41df-bedd-d678c548dbc1 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:58:56.014418 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a402e5d-4cf7-49e9-9ea1-9b1506331f4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:58:56.015029 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a402e5d-4cf7-49e9-9ea1-9b1506331f4e service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:56.015301 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a402e5d-4cf7-49e9-9ea1-9b1506331f4e service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:56.015563 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a402e5d-4cf7-49e9-9ea1-9b1506331f4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:56.017203 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-6a402e5d-4cf7-49e9-9ea1-9b1506331f4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:56.065477 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:56.106309 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a402e5d-4cf7-49e9-9ea1-9b1506331f4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:58:56.107888 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a402e5d-4cf7-49e9-9ea1-9b1506331f4e service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:58:56.126164 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 715/1428] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:58:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:58:57.217348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54644faa-5d79-4b2d-8d60-4bce5adfc4ba None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:57.217936 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54644faa-5d79-4b2d-8d60-4bce5adfc4ba None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:57.217936 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54644faa-5d79-4b2d-8d60-4bce5adfc4ba None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:57.219239 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 20:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:58:57.225573 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf567ff-7804-4f57-a456-feb9456863b7 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:57.226068 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf567ff-7804-4f57-a456-feb9456863b7 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:57.226165 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf567ff-7804-4f57-a456-feb9456863b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:57.301771 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-bdf567ff-7804-4f57-a456-feb9456863b7 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:58:57.302887 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-bdf567ff-7804-4f57-a456-feb9456863b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:57.397250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 20:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:58:57.546297 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54644faa-5d79-4b2d-8d60-4bce5adfc4ba None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:58:57.547876 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-54644faa-5d79-4b2d-8d60-4bce5adfc4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:57.596154 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:57.691540 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54644faa-5d79-4b2d-8d60-4bce5adfc4ba None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:58:57.802865 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c019a06-221b-4428-8e31-19cacb058f49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:58:57.803473 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c019a06-221b-4428-8e31-19cacb058f49 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:57.803761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c019a06-221b-4428-8e31-19cacb058f49 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:57.804029 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c019a06-221b-4428-8e31-19cacb058f49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:57.805655 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0c019a06-221b-4428-8e31-19cacb058f49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:57.812556 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:57.895025 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c019a06-221b-4428-8e31-19cacb058f49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:58:57.896590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c019a06-221b-4428-8e31-19cacb058f49 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:58:57.914939 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 715/1431] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:58:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:58:59.079620 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d2d3d-c483-4292-950c-61e652d70222 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:59.080168 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d2d3d-c483-4292-950c-61e652d70222 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:59.080265 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d2d3d-c483-4292-950c-61e652d70222 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:59.081593 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 20:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:58:59.087787 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc1482-84a9-4589-a6e1-7ab9ca32d310 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:59.088279 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc1482-84a9-4589-a6e1-7ab9ca32d310 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:59.088384 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc1482-84a9-4589-a6e1-7ab9ca32d310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:59.161247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-37cc1482-84a9-4589-a6e1-7ab9ca32d310 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:58:59.162439 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-37cc1482-84a9-4589-a6e1-7ab9ca32d310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:59.227891 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:59.256533 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 20:58:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:58:59.413177 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d2d3d-c483-4292-950c-61e652d70222 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:58:59.414689 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-369d2d3d-c483-4292-950c-61e652d70222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:59.432623 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:59.559575 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d2d3d-c483-4292-950c-61e652d70222 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:58:59.765368 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:58:59.790998 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda38e4-5a34-4194-8120-e04062c710d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:58:59.791577 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda38e4-5a34-4194-8120-e04062c710d4 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:58:59.791853 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda38e4-5a34-4194-8120-e04062c710d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:58:59.792152 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda38e4-5a34-4194-8120-e04062c710d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:58:59.793735 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-fdda38e4-5a34-4194-8120-e04062c710d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:58:59.881060 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdda38e4-5a34-4194-8120-e04062c710d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:58:59.882661 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdda38e4-5a34-4194-8120-e04062c710d4 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:58:59.900694 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 718/1434] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:58:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:59:01.086381 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7937de4-002a-4e15-aed4-a9549a2e42c0 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:01.086925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7937de4-002a-4e15-aed4-a9549a2e42c0 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:01.086925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7937de4-002a-4e15-aed4-a9549a2e42c0 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:01.088354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 20:59:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:59:01.095193 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ecdb1-62a9-4515-bb5a-133bf73b9e84 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:01.095599 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ecdb1-62a9-4515-bb5a-133bf73b9e84 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:01.095710 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ecdb1-62a9-4515-bb5a-133bf73b9e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:01.169796 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-8c1ecdb1-62a9-4515-bb5a-133bf73b9e84 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:59:01.170992 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8c1ecdb1-62a9-4515-bb5a-133bf73b9e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:01.211595 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:59:01.265281 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 20:59:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:59:01.418277 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7937de4-002a-4e15-aed4-a9549a2e42c0 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:59:01.419785 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a7937de4-002a-4e15-aed4-a9549a2e42c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:01.561754 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7937de4-002a-4e15-aed4-a9549a2e42c0 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:59:01.615316 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:59:01.672362 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c684a-f881-4357-9166-2169d2cf15e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:59:01.672968 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c684a-f881-4357-9166-2169d2cf15e4 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:01.673251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c684a-f881-4357-9166-2169d2cf15e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:01.673522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c684a-f881-4357-9166-2169d2cf15e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:01.675125 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1f8c684a-f881-4357-9166-2169d2cf15e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:01.763589 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c684a-f881-4357-9166-2169d2cf15e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:59:01.765222 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c684a-f881-4357-9166-2169d2cf15e4 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:59:01.783404 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 718/1437] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:59:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:59:02.885877 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90cd6e3-e38c-46f0-8ad6-f6dfe38f747d None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:02.892242 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90cd6e3-e38c-46f0-8ad6-f6dfe38f747d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:02.892242 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90cd6e3-e38c-46f0-8ad6-f6dfe38f747d None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:02.892242 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 20:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:59:02.894450 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff232b-3362-4809-8573-3dca03ab2ec9 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:02.894943 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff232b-3362-4809-8573-3dca03ab2ec9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:02.895068 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff232b-3362-4809-8573-3dca03ab2ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:02.976690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-7dff232b-3362-4809-8573-3dca03ab2ec9 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:59:02.977890 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7dff232b-3362-4809-8573-3dca03ab2ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:02.980564 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:59:03.074934 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 20:59:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:59:03.227922 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90cd6e3-e38c-46f0-8ad6-f6dfe38f747d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:59:03.230142 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e90cd6e3-e38c-46f0-8ad6-f6dfe38f747d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:03.375478 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90cd6e3-e38c-46f0-8ad6-f6dfe38f747d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:59:03.402678 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:59:03.485305 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb22fc-7439-4ad3-8df6-4a8e420a6d3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:59:03.485828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb22fc-7439-4ad3-8df6-4a8e420a6d3b service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:03.486100 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb22fc-7439-4ad3-8df6-4a8e420a6d3b service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:03.486363 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb22fc-7439-4ad3-8df6-4a8e420a6d3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:03.487931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d9eb22fc-7439-4ad3-8df6-4a8e420a6d3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:03.581170 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb22fc-7439-4ad3-8df6-4a8e420a6d3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:59:03.582742 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb22fc-7439-4ad3-8df6-4a8e420a6d3b service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:59:03.600814 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 721/1440] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:59:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:59:05.671589 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10f898-6858-4aed-b9b7-545b5c6736da None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:05.672145 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10f898-6858-4aed-b9b7-545b5c6736da None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:05.672238 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10f898-6858-4aed-b9b7-545b5c6736da None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:05.673528 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 20:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:59:05.678746 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e9c58-36c7-4842-a305-31d20e26eb27 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:05.679198 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e9c58-36c7-4842-a305-31d20e26eb27 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:05.679306 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e9c58-36c7-4842-a305-31d20e26eb27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:05.719784 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:59:05.744149 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-3d5e9c58-36c7-4842-a305-31d20e26eb27 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:59:05.745415 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3d5e9c58-36c7-4842-a305-31d20e26eb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:05.840161 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 20:59:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:59:06.127957 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10f898-6858-4aed-b9b7-545b5c6736da None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:59:06.129522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1f10f898-6858-4aed-b9b7-545b5c6736da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:06.231338 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10f898-6858-4aed-b9b7-545b5c6736da None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:59:06.263828 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:59:06.303792 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68558013-1c8e-4e43-99d8-b14a44581c98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:59:06.304417 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68558013-1c8e-4e43-99d8-b14a44581c98 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:06.304695 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68558013-1c8e-4e43-99d8-b14a44581c98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:06.304963 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68558013-1c8e-4e43-99d8-b14a44581c98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:06.306540 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-68558013-1c8e-4e43-99d8-b14a44581c98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:06.531873 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68558013-1c8e-4e43-99d8-b14a44581c98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:59:06.533362 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68558013-1c8e-4e43-99d8-b14a44581c98 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:59:06.551673 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 721/1443] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:59:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:59:08.832613 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07230a21-9f0f-4464-a116-33ce20f60ff4 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:08.833190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07230a21-9f0f-4464-a116-33ce20f60ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:08.833190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07230a21-9f0f-4464-a116-33ce20f60ff4 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:08.834553 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 20:59:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:59:08.839754 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b012f87-7dd9-4c8b-8168-a428d5511bd4 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:08.840233 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b012f87-7dd9-4c8b-8168-a428d5511bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:08.840332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b012f87-7dd9-4c8b-8168-a428d5511bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:08.904690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-2b012f87-7dd9-4c8b-8168-a428d5511bd4 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:59:08.905890 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2b012f87-7dd9-4c8b-8168-a428d5511bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:08.918938 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:59:09.003119 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 20:59:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:59:09.289682 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07230a21-9f0f-4464-a116-33ce20f60ff4 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:59:09.291277 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-07230a21-9f0f-4464-a116-33ce20f60ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:09.393665 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07230a21-9f0f-4464-a116-33ce20f60ff4 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:59:09.406160 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:59:09.465762 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ef44e-9f97-482e-8800-1ed3e475cb1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:59:09.466363 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ef44e-9f97-482e-8800-1ed3e475cb1b service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:09.466638 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ef44e-9f97-482e-8800-1ed3e475cb1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:09.466906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ef44e-9f97-482e-8800-1ed3e475cb1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:09.468501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-9c4ef44e-9f97-482e-8800-1ed3e475cb1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:09.575524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ef44e-9f97-482e-8800-1ed3e475cb1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:59:09.576982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ef44e-9f97-482e-8800-1ed3e475cb1b service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:59:09.594605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 724/1446] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:59:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:59:12.132866 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f81636-8e43-40be-834f-f561b226a0c4 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:12.133401 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f81636-8e43-40be-834f-f561b226a0c4 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:12.133401 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f81636-8e43-40be-834f-f561b226a0c4 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:12.134773 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 20:59:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:59:12.140716 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943ca43-ce6e-444e-a01e-6566194029fe None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:12.141130 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943ca43-ce6e-444e-a01e-6566194029fe None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:12.141258 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943ca43-ce6e-444e-a01e-6566194029fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:12.187837 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:59:12.206503 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-d943ca43-ce6e-444e-a01e-6566194029fe None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:59:12.207710 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d943ca43-ce6e-444e-a01e-6566194029fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:12.306425 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 20:59:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:59:12.589749 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f81636-8e43-40be-834f-f561b226a0c4 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:59:12.591268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-60f81636-8e43-40be-834f-f561b226a0c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:12.702123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f81636-8e43-40be-834f-f561b226a0c4 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:59:12.742074 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:59:12.775026 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616aab4a-afcd-4582-8b39-9ee203e91420 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:59:12.776167 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616aab4a-afcd-4582-8b39-9ee203e91420 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:12.776446 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616aab4a-afcd-4582-8b39-9ee203e91420 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:12.776712 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616aab4a-afcd-4582-8b39-9ee203e91420 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:12.778296 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-616aab4a-afcd-4582-8b39-9ee203e91420 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:12.891585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616aab4a-afcd-4582-8b39-9ee203e91420 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:59:12.891585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616aab4a-afcd-4582-8b39-9ee203e91420 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:59:12.905283 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 724/1449] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:59:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:59:15.213376 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aafad6-d664-4135-be85-a9b0b727dd0c None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:15.213962 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aafad6-d664-4135-be85-a9b0b727dd0c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:15.213962 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aafad6-d664-4135-be85-a9b0b727dd0c None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:15.215257 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 20:59:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:59:15.220574 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da491877-6500-4264-833b-339e2a62477d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:15.220982 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da491877-6500-4264-833b-339e2a62477d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:15.221116 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da491877-6500-4264-833b-339e2a62477d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:15.290670 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-da491877-6500-4264-833b-339e2a62477d None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:59:15.292517 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-da491877-6500-4264-833b-339e2a62477d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:15.312693 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:59:15.386993 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 20:59:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:59:15.677051 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aafad6-d664-4135-be85-a9b0b727dd0c None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:59:15.678620 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-21aafad6-d664-4135-be85-a9b0b727dd0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:15.781833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aafad6-d664-4135-be85-a9b0b727dd0c None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:59:15.801972 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:59:15.855542 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b2639-8140-4102-a4b8-68ac6513aa9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:59:15.856195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b2639-8140-4102-a4b8-68ac6513aa9e service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:15.856473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b2639-8140-4102-a4b8-68ac6513aa9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:15.856741 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b2639-8140-4102-a4b8-68ac6513aa9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:15.858310 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-084b2639-8140-4102-a4b8-68ac6513aa9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:15.967379 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084b2639-8140-4102-a4b8-68ac6513aa9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:59:15.968832 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084b2639-8140-4102-a4b8-68ac6513aa9e service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:59:15.987198 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 727/1452] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:59:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:59:18.188941 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241a730-320e-433e-a2f3-0dfe1f9562ba None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:18.189490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241a730-320e-433e-a2f3-0dfe1f9562ba None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:18.189700 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241a730-320e-433e-a2f3-0dfe1f9562ba None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:18.191070 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 20:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:59:18.198382 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6366b44-11e1-40a5-9dc6-64d3963f41ef None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:18.198878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6366b44-11e1-40a5-9dc6-64d3963f41ef None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:18.198975 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6366b44-11e1-40a5-9dc6-64d3963f41ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:18.261829 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:59:18.264620 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-e6366b44-11e1-40a5-9dc6-64d3963f41ef None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:59:18.265838 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e6366b44-11e1-40a5-9dc6-64d3963f41ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:18.368448 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 20:59:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 20:59:18.661431 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d241a730-320e-433e-a2f3-0dfe1f9562ba None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:59:18.663029 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d241a730-320e-433e-a2f3-0dfe1f9562ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:18.767590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d241a730-320e-433e-a2f3-0dfe1f9562ba None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:59:18.816250 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:59:18.839907 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b17479-0be2-4786-a49a-8eab89bc4a52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:59:18.840542 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b17479-0be2-4786-a49a-8eab89bc4a52 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:18.840823 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b17479-0be2-4786-a49a-8eab89bc4a52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:18.841096 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b17479-0be2-4786-a49a-8eab89bc4a52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:18.842666 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-60b17479-0be2-4786-a49a-8eab89bc4a52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:18.958262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b17479-0be2-4786-a49a-8eab89bc4a52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:59:18.959713 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b17479-0be2-4786-a49a-8eab89bc4a52 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:59:18.977805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 727/1455] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:59:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 20:59:21.175295 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6dada3-b0cb-46b4-9580-efa1038e17bc None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:21.175809 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6dada3-b0cb-46b4-9580-efa1038e17bc None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:21.175905 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6dada3-b0cb-46b4-9580-efa1038e17bc None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:21.177229 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 20:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:59:21.182622 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42d795-df1f-4875-b27a-ef8b90bb2ad2 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:21.183048 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42d795-df1f-4875-b27a-ef8b90bb2ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:21.183170 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42d795-df1f-4875-b27a-ef8b90bb2ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:21.248248 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-be42d795-df1f-4875-b27a-ef8b90bb2ad2 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:59:21.249449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-be42d795-df1f-4875-b27a-ef8b90bb2ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:21.277961 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:59:21.344780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 20:59:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:59:21.626707 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6dada3-b0cb-46b4-9580-efa1038e17bc None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:59:21.628272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-db6dada3-b0cb-46b4-9580-efa1038e17bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:21.730625 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6dada3-b0cb-46b4-9580-efa1038e17bc None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:59:21.760477 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:59:21.804518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab1a93-8e4a-4475-95f2-649b018215c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:59:21.805057 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab1a93-8e4a-4475-95f2-649b018215c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:21.805363 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab1a93-8e4a-4475-95f2-649b018215c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:21.805638 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab1a93-8e4a-4475-95f2-649b018215c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:21.807737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-90ab1a93-8e4a-4475-95f2-649b018215c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:21.915380 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab1a93-8e4a-4475-95f2-649b018215c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:59:21.916867 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab1a93-8e4a-4475-95f2-649b018215c7 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:59:21.935711 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 730/1458] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:59:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:59:24.151627 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ec25d-621e-4f11-ad70-2ac1cbedb065 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:24.152260 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ec25d-621e-4f11-ad70-2ac1cbedb065 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:24.152260 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ec25d-621e-4f11-ad70-2ac1cbedb065 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:24.153555 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 20:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:59:24.159037 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c85ed1-1929-458f-9686-4aca29a97b8d None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:24.159446 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c85ed1-1929-458f-9686-4aca29a97b8d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:24.159558 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c85ed1-1929-458f-9686-4aca29a97b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:24.224404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-04c85ed1-1929-458f-9686-4aca29a97b8d None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:59:24.225594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-04c85ed1-1929-458f-9686-4aca29a97b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:24.233830 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:59:24.321823 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 20:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:59:24.613737 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ec25d-621e-4f11-ad70-2ac1cbedb065 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:59:24.615267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-006ec25d-621e-4f11-ad70-2ac1cbedb065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:24.725001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ec25d-621e-4f11-ad70-2ac1cbedb065 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:59:24.786257 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:59:24.804600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9734d6c1-2494-4f00-b305-b79f4edded9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:59:24.805201 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734d6c1-2494-4f00-b305-b79f4edded9b service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:24.805477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734d6c1-2494-4f00-b305-b79f4edded9b service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:24.805747 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734d6c1-2494-4f00-b305-b79f4edded9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:24.807354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9734d6c1-2494-4f00-b305-b79f4edded9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:24.914314 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9734d6c1-2494-4f00-b305-b79f4edded9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:59:24.916372 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9734d6c1-2494-4f00-b305-b79f4edded9b service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:59:24.935336 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 730/1461] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:59:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 20:59:27.125651 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfa4a4-4180-408f-8eb2-10f430fc052a None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:27.126216 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfa4a4-4180-408f-8eb2-10f430fc052a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:27.126216 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfa4a4-4180-408f-8eb2-10f430fc052a None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:27.127626 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 20:59:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:59:27.132984 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bf4c7-0655-464a-9877-2bbee0d89d24 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:27.133405 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bf4c7-0655-464a-9877-2bbee0d89d24 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:27.133539 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bf4c7-0655-464a-9877-2bbee0d89d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:27.197976 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-229bf4c7-0655-464a-9877-2bbee0d89d24 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:59:27.199172 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-229bf4c7-0655-464a-9877-2bbee0d89d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:27.236770 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:59:27.293177 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 20:59:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:59:27.585308 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdfa4a4-4180-408f-8eb2-10f430fc052a None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:59:27.586900 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-dfdfa4a4-4180-408f-8eb2-10f430fc052a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:27.692132 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdfa4a4-4180-408f-8eb2-10f430fc052a None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:59:27.730671 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:59:27.767299 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b4061-72d9-4572-b879-12e9109fdf6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:59:27.767935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b4061-72d9-4572-b879-12e9109fdf6a service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:27.768259 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b4061-72d9-4572-b879-12e9109fdf6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:27.768527 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b4061-72d9-4572-b879-12e9109fdf6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:27.770109 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-471b4061-72d9-4572-b879-12e9109fdf6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:27.876393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b4061-72d9-4572-b879-12e9109fdf6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:59:27.877744 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b4061-72d9-4572-b879-12e9109fdf6a service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:59:27.896387 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 733/1464] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:59:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:59:30.091603 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5bfd8-0549-4e25-85ce-ad994fa55f5e None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:30.092264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5bfd8-0549-4e25-85ce-ad994fa55f5e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:30.092264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5bfd8-0549-4e25-85ce-ad994fa55f5e None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:30.093509 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 20:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:59:30.099049 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4131a6-b5e9-4ff0-898a-645d77774879 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:30.099490 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4131a6-b5e9-4ff0-898a-645d77774879 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:30.099612 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4131a6-b5e9-4ff0-898a-645d77774879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:30.169688 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-3c4131a6-b5e9-4ff0-898a-645d77774879 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:59:30.170875 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3c4131a6-b5e9-4ff0-898a-645d77774879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:30.188197 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:59:30.263587 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 20:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:59:30.543506 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5bfd8-0549-4e25-85ce-ad994fa55f5e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:59:30.545068 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5fb5bfd8-0549-4e25-85ce-ad994fa55f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:30.645465 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5bfd8-0549-4e25-85ce-ad994fa55f5e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:59:30.720756 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:59:30.727435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5e33f-e209-45f0-a4d4-8aff07e9a4f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:59:30.728080 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5e33f-e209-45f0-a4d4-8aff07e9a4f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:30.728370 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5e33f-e209-45f0-a4d4-8aff07e9a4f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:30.728637 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5e33f-e209-45f0-a4d4-8aff07e9a4f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:30.730230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5cb5e33f-e209-45f0-a4d4-8aff07e9a4f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:30.838029 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb5e33f-e209-45f0-a4d4-8aff07e9a4f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:59:30.839395 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb5e33f-e209-45f0-a4d4-8aff07e9a4f2 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:59:30.858702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 733/1467] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:59:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:59:33.045139 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933b7fb-b9da-4f94-8691-2f205fe16170 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:33.045705 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933b7fb-b9da-4f94-8691-2f205fe16170 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:33.045705 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933b7fb-b9da-4f94-8691-2f205fe16170 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:33.047081 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 20:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:59:33.052448 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a8467-976f-47ae-bfdd-2ba26ad95d15 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:33.052888 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a8467-976f-47ae-bfdd-2ba26ad95d15 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:33.053047 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a8467-976f-47ae-bfdd-2ba26ad95d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:33.118072 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-2d2a8467-976f-47ae-bfdd-2ba26ad95d15 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:59:33.119367 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2d2a8467-976f-47ae-bfdd-2ba26ad95d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:33.166920 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:59:33.214303 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 20:59:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:59:33.502017 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933b7fb-b9da-4f94-8691-2f205fe16170 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:59:33.503585 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-9933b7fb-b9da-4f94-8691-2f205fe16170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:33.607185 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933b7fb-b9da-4f94-8691-2f205fe16170 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:59:33.654431 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:59:33.678811 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b1e03-cf8c-4759-9c59-d6edf0aba4d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:59:33.679412 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b1e03-cf8c-4759-9c59-d6edf0aba4d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:33.679691 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b1e03-cf8c-4759-9c59-d6edf0aba4d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:33.679959 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b1e03-cf8c-4759-9c59-d6edf0aba4d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:33.681809 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-bb6b1e03-cf8c-4759-9c59-d6edf0aba4d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:33.808118 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6b1e03-cf8c-4759-9c59-d6edf0aba4d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:59:33.809465 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6b1e03-cf8c-4759-9c59-d6edf0aba4d0 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:59:33.827678 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 736/1470] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:59:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:59:43.212683 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=22 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:59:43.825533 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=66 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:59:53.775511 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f1147-1524-4c80-bead-5fca02110883 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:53.776046 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f1147-1524-4c80-bead-5fca02110883 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:53.776046 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f1147-1524-4c80-bead-5fca02110883 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:53.777464 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 735/1471] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:59:53.782778 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d5908-5ae2-497b-b0d9-a91245cab2e5 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:53.783244 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d5908-5ae2-497b-b0d9-a91245cab2e5 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:53.783352 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d5908-5ae2-497b-b0d9-a91245cab2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:53.849256 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-339d5908-5ae2-497b-b0d9-a91245cab2e5 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:59:53.850377 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-339d5908-5ae2-497b-b0d9-a91245cab2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:53.949011 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 737/1472] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:59:54.248954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322af3b5-c14c-4322-9c81-e40bc4d3856e None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:54.249237 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322af3b5-c14c-4322-9c81-e40bc4d3856e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:54.249507 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322af3b5-c14c-4322-9c81-e40bc4d3856e None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:54.250812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 736/1473] 10.222.0.48 () {54 vars in 966 bytes} [Fri Sep 9 20:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:59:54.255672 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632b346-e22c-45aa-8117-05ea7b53f3c2 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:54.255962 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632b346-e22c-45aa-8117-05ea7b53f3c2 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:54.256267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632b346-e22c-45aa-8117-05ea7b53f3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:54.331469 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-2632b346-e22c-45aa-8117-05ea7b53f3c2 None None] MFA Rules not processed for user `7832e97d7e97439e821268d05816ae3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:59:54.332693 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2632b346-e22c-45aa-8117-05ea7b53f3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:54.353128 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:59:54.407340 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 738/1474] 10.222.0.48 () {58 vars in 1063 bytes} [Fri Sep 9 20:59:54 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:59:54.415071 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322af3b5-c14c-4322-9c81-e40bc4d3856e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:59:54.416605 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-322af3b5-c14c-4322-9c81-e40bc4d3856e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:54.518261 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322af3b5-c14c-4322-9c81-e40bc4d3856e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:59:54.592397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e9f745-0326-48c4-9781-9b33a69dc031 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7832e97d7e97439e821268d05816ae3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:59:54.593002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9f745-0326-48c4-9781-9b33a69dc031 service glance] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:54.593284 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9f745-0326-48c4-9781-9b33a69dc031 service glance] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:54.593557 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9f745-0326-48c4-9781-9b33a69dc031 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:54.595154 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a2e9f745-0326-48c4-9781-9b33a69dc031 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:54.648747 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:59:54.703334 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e9f745-0326-48c4-9781-9b33a69dc031 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:59:54.704849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e9f745-0326-48c4-9781-9b33a69dc031 service glance] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:59:54.724454 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 737/1475] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 20:59:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 20:59:56.944004 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa6e16-d39d-469c-945a-9975b35aa7e8 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:56.944891 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa6e16-d39d-469c-945a-9975b35aa7e8 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:56.944891 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa6e16-d39d-469c-945a-9975b35aa7e8 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:56.946187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 739/1476] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:59:56.951728 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ab6d5-a7a6-4f05-b841-6d075d3df3b2 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:56.956892 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ab6d5-a7a6-4f05-b841-6d075d3df3b2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:56.956892 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ab6d5-a7a6-4f05-b841-6d075d3df3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:57.017528 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-cb1ab6d5-a7a6-4f05-b841-6d075d3df3b2 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:59:57.018719 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cb1ab6d5-a7a6-4f05-b841-6d075d3df3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:57.112481 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 738/1477] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:59:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 20:59:57.383420 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa6e16-d39d-469c-945a-9975b35aa7e8 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 20:59:57.384977 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-08fa6e16-d39d-469c-945a-9975b35aa7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:57.508554 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa6e16-d39d-469c-945a-9975b35aa7e8 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 20:59:57.557232 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 20:59:57.600243 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4e61d-ea1a-4d7e-b1c7-41f22d767d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 20:59:57.600842 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4e61d-ea1a-4d7e-b1c7-41f22d767d7a admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:57.601118 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4e61d-ea1a-4d7e-b1c7-41f22d767d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:57.601380 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4e61d-ea1a-4d7e-b1c7-41f22d767d7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:57.602010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4e61d-ea1a-4d7e-b1c7-41f22d767d7a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 20:59:57.604297 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: WARNING py.warnings [None req-8ee4e61d-ea1a-4d7e-b1c7-41f22d767d7a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 20:59:57.604297 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: warnings.warn(msg) Sep 09 20:59:57.604297 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: Sep 09 20:59:57.605415 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4e61d-ea1a-4d7e-b1c7-41f22d767d7a admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 20:59:57.624292 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 740/1478] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Sep 9 20:59:57 2022] GET /identity/v3/projects => generated 2553 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 20:59:59.793119 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e1b09-0962-4f41-857b-0fef032145fe None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:59.793658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e1b09-0962-4f41-857b-0fef032145fe None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:59.793658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e1b09-0962-4f41-857b-0fef032145fe None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:59.795041 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 739/1479] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 20:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 20:59:59.800508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f8c3e-7931-4282-a9c9-ca11aa6b475a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 20:59:59.800944 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f8c3e-7931-4282-a9c9-ca11aa6b475a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 20:59:59.801049 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f8c3e-7931-4282-a9c9-ca11aa6b475a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 20:59:59.865914 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-cf0f8c3e-7931-4282-a9c9-ca11aa6b475a None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 20:59:59.867040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-cf0f8c3e-7931-4282-a9c9-ca11aa6b475a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 20:59:59.961736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 741/1480] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 20:59:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:00:00.254103 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e1b09-0962-4f41-857b-0fef032145fe None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:00:00.255654 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-517e1b09-0962-4f41-857b-0fef032145fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:00:00.266326 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:00:00.399478 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e1b09-0962-4f41-857b-0fef032145fe None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:00:00.476778 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:00:00.519214 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254851cb-3042-4ff0-9527-886fa17b2dc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:00:00.519722 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254851cb-3042-4ff0-9527-886fa17b2dc3 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:00:00.520004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254851cb-3042-4ff0-9527-886fa17b2dc3 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:00:00.520308 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254851cb-3042-4ff0-9527-886fa17b2dc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:00:00.521906 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-254851cb-3042-4ff0-9527-886fa17b2dc3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:00:00.610699 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254851cb-3042-4ff0-9527-886fa17b2dc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:00:00.612240 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254851cb-3042-4ff0-9527-886fa17b2dc3 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:00:00.630415 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 740/1481] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:00:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:00:02.971489 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1631aae-0d42-4b24-969a-301f935a05d5 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:00:02.972093 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1631aae-0d42-4b24-969a-301f935a05d5 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:00:02.972093 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1631aae-0d42-4b24-969a-301f935a05d5 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:00:02.973411 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 742/1482] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 21:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:00:02.978904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47cc79-3f2c-4dd7-8003-f00999a5a18d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:00:02.979358 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47cc79-3f2c-4dd7-8003-f00999a5a18d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:00:02.979465 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47cc79-3f2c-4dd7-8003-f00999a5a18d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:00:03.049585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5b47cc79-3f2c-4dd7-8003-f00999a5a18d None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:00:03.050794 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5b47cc79-3f2c-4dd7-8003-f00999a5a18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:00:03.096148 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:00:03.145841 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 741/1483] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 21:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:00:03.431741 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1631aae-0d42-4b24-969a-301f935a05d5 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:00:03.433391 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c1631aae-0d42-4b24-969a-301f935a05d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:00:03.461501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:00:03.540391 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1631aae-0d42-4b24-969a-301f935a05d5 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:00:03.615647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ebffa-b7b6-40eb-8ee6-eafcf72f85be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:00:03.616274 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ebffa-b7b6-40eb-8ee6-eafcf72f85be service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:00:03.616551 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ebffa-b7b6-40eb-8ee6-eafcf72f85be service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:00:03.616819 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ebffa-b7b6-40eb-8ee6-eafcf72f85be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:00:03.618396 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-344ebffa-b7b6-40eb-8ee6-eafcf72f85be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:00:03.691265 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:00:03.727396 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ebffa-b7b6-40eb-8ee6-eafcf72f85be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:00:03.729037 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ebffa-b7b6-40eb-8ee6-eafcf72f85be service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:00:03.747974 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 743/1484] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:00:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:00:05.978680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b3b3a-42b6-4bd7-bb3b-a66d77cbda99 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:00:05.979250 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b3b3a-42b6-4bd7-bb3b-a66d77cbda99 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:00:05.979250 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b3b3a-42b6-4bd7-bb3b-a66d77cbda99 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:00:05.980672 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 742/1485] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 21:00:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:00:05.986392 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77c9f3-8a03-4a64-9b53-e418eb8f8c55 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:00:05.986876 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77c9f3-8a03-4a64-9b53-e418eb8f8c55 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:00:05.986970 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77c9f3-8a03-4a64-9b53-e418eb8f8c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:00:06.052850 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-8e77c9f3-8a03-4a64-9b53-e418eb8f8c55 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:00:06.054061 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8e77c9f3-8a03-4a64-9b53-e418eb8f8c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:00:06.147511 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 744/1486] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 21:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:00:06.436648 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b3b3a-42b6-4bd7-bb3b-a66d77cbda99 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:00:06.438166 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ec0b3b3a-42b6-4bd7-bb3b-a66d77cbda99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:00:06.540262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b3b3a-42b6-4bd7-bb3b-a66d77cbda99 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:00:06.611412 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:00:06.612927 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4b270-1475-4148-aa3f-b51ce614192f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:00:06.613529 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4b270-1475-4148-aa3f-b51ce614192f service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:00:06.613811 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4b270-1475-4148-aa3f-b51ce614192f service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:00:06.614084 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4b270-1475-4148-aa3f-b51ce614192f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:00:06.615678 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a9e4b270-1475-4148-aa3f-b51ce614192f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:00:06.722109 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e4b270-1475-4148-aa3f-b51ce614192f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:00:06.723460 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e4b270-1475-4148-aa3f-b51ce614192f service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:00:06.741111 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 743/1487] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:00:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:00:09.026432 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0b197-a14d-4b5d-90b3-4ab4a4252b83 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:00:09.026879 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0b197-a14d-4b5d-90b3-4ab4a4252b83 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:00:09.026998 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0b197-a14d-4b5d-90b3-4ab4a4252b83 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:00:09.028366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 745/1488] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 21:00:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:00:09.033758 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0592e-7aa2-4e15-965e-71b091cbb320 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:00:09.034204 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0592e-7aa2-4e15-965e-71b091cbb320 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:00:09.034304 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0592e-7aa2-4e15-965e-71b091cbb320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:00:09.099388 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-0ea0592e-7aa2-4e15-965e-71b091cbb320 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:00:09.100606 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0ea0592e-7aa2-4e15-965e-71b091cbb320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:00:09.152954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:00:09.197299 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 744/1489] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 21:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:00:09.487473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a0b197-a14d-4b5d-90b3-4ab4a4252b83 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:00:09.489042 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d4a0b197-a14d-4b5d-90b3-4ab4a4252b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:00:09.524474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:00:09.591534 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a0b197-a14d-4b5d-90b3-4ab4a4252b83 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:00:09.663538 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ce3be0-5cc4-425d-b1ba-b332c0ee5bb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:00:09.664162 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce3be0-5cc4-425d-b1ba-b332c0ee5bb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:00:09.664440 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce3be0-5cc4-425d-b1ba-b332c0ee5bb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:00:09.664708 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce3be0-5cc4-425d-b1ba-b332c0ee5bb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:00:09.666304 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b4ce3be0-5cc4-425d-b1ba-b332c0ee5bb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:00:09.746306 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:00:09.774734 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ce3be0-5cc4-425d-b1ba-b332c0ee5bb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:00:09.776927 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ce3be0-5cc4-425d-b1ba-b332c0ee5bb8 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:00:09.794601 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 746/1490] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:00:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:00:12.335509 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e429c-3116-420b-a644-3567d683f690 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:00:12.336025 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e429c-3116-420b-a644-3567d683f690 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:00:12.336025 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e429c-3116-420b-a644-3567d683f690 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:00:12.337986 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 745/1491] 10.222.0.48 () {54 vars in 927 bytes} [Fri Sep 9 21:00:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:00:12.343376 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb27f9-1f10-4b60-8ce4-e13a836df0b2 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:00:12.343774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb27f9-1f10-4b60-8ce4-e13a836df0b2 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:00:12.343917 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb27f9-1f10-4b60-8ce4-e13a836df0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:00:12.410314 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-3cdb27f9-1f10-4b60-8ce4-e13a836df0b2 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:00:12.411523 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3cdb27f9-1f10-4b60-8ce4-e13a836df0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:00:12.505731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 747/1492] 10.222.0.48 () {58 vars in 1024 bytes} [Fri Sep 9 21:00:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:00:12.798771 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e429c-3116-420b-a644-3567d683f690 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:00:12.800337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-605e429c-3116-420b-a644-3567d683f690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:00:12.905002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e429c-3116-420b-a644-3567d683f690 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:00:12.982840 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449f7a3-8862-4701-96a9-ac7ace8e6b0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:00:12.983457 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449f7a3-8862-4701-96a9-ac7ace8e6b0a service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:00:12.983737 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449f7a3-8862-4701-96a9-ac7ace8e6b0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:00:12.984010 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449f7a3-8862-4701-96a9-ac7ace8e6b0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:00:12.985656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b449f7a3-8862-4701-96a9-ac7ace8e6b0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:00:12.994892 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:00:13.094870 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b449f7a3-8862-4701-96a9-ac7ace8e6b0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:00:13.096269 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b449f7a3-8862-4701-96a9-ac7ace8e6b0a service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:00:13.114676 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 746/1493] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:00:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:00:22.504210 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=82 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:00:23.138453 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=50 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:01:44.526350 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b124c9-0a08-44b4-8dde-cb8a7c71ae31 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:01:44.526947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b124c9-0a08-44b4-8dde-cb8a7c71ae31 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:01:44.526947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b124c9-0a08-44b4-8dde-cb8a7c71ae31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:01:44.602923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-26b124c9-0a08-44b4-8dde-cb8a7c71ae31 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:01:44.604208 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-26b124c9-0a08-44b4-8dde-cb8a7c71ae31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:01:44.701408 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 748/1494] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:01:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:01:44.709409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c5f18-4f70-479a-bc99-eaa922441d4e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:01:44.709895 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c5f18-4f70-479a-bc99-eaa922441d4e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:01:44.710018 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c5f18-4f70-479a-bc99-eaa922441d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:01:44.786801 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b37c5f18-4f70-479a-bc99-eaa922441d4e None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:01:44.787940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b37c5f18-4f70-479a-bc99-eaa922441d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:01:44.886904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 747/1495] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:01:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:01:44.891548 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b124c9-0a08-44b4-8dde-cb8a7c71ae31 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:01:44.892795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-26b124c9-0a08-44b4-8dde-cb8a7c71ae31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:01:44.904522 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b124c9-0a08-44b4-8dde-cb8a7c71ae31 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:01:44.965993 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:01:44.999335 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f91c4a-274a-42c4-bc2e-9a2cff2a03f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:01:44.999940 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f91c4a-274a-42c4-bc2e-9a2cff2a03f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:01:45.000254 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f91c4a-274a-42c4-bc2e-9a2cff2a03f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:01:45.000526 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f91c4a-274a-42c4-bc2e-9a2cff2a03f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:01:45.001160 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f91c4a-274a-42c4-bc2e-9a2cff2a03f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:01:45.001461 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f91c4a-274a-42c4-bc2e-9a2cff2a03f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:01:45.002459 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: WARNING py.warnings [None req-33f91c4a-274a-42c4-bc2e-9a2cff2a03f7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 21:01:45.002459 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: warnings.warn(msg) Sep 09 21:01:45.002459 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: Sep 09 21:01:45.003338 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f91c4a-274a-42c4-bc2e-9a2cff2a03f7 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:01:45.015142 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 749/1496] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:01:45.023267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c5f18-4f70-479a-bc99-eaa922441d4e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:01:45.032237 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c5f18-4f70-479a-bc99-eaa922441d4e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:01:45.092206 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:01:45.124903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd51a27-fc37-4642-a728-6e7b256205f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:01:45.125498 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd51a27-fc37-4642-a728-6e7b256205f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:01:45.125775 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd51a27-fc37-4642-a728-6e7b256205f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:01:45.126042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd51a27-fc37-4642-a728-6e7b256205f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:01:45.126672 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd51a27-fc37-4642-a728-6e7b256205f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:01:45.128933 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: WARNING py.warnings [None req-8fd51a27-fc37-4642-a728-6e7b256205f8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 21:01:45.128933 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: warnings.warn(msg) Sep 09 21:01:45.128933 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: Sep 09 21:01:45.130482 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd51a27-fc37-4642-a728-6e7b256205f8 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:01:45.238093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 748/1497] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:01:45 2022] POST /identity/v3/projects => generated 373 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:01:45.242373 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f91c4a-274a-42c4-bc2e-9a2cff2a03f7 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:01:45.244450 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-33f91c4a-274a-42c4-bc2e-9a2cff2a03f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:01:45.277443 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f91c4a-274a-42c4-bc2e-9a2cff2a03f7 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:01:45.373102 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a6cd0-b17d-4fa6-a096-99f5f9c8ba8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:01:45.373623 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a6cd0-b17d-4fa6-a096-99f5f9c8ba8c admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:01:45.373900 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a6cd0-b17d-4fa6-a096-99f5f9c8ba8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:01:45.374169 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a6cd0-b17d-4fa6-a096-99f5f9c8ba8c admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:01:45.374799 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a6cd0-b17d-4fa6-a096-99f5f9c8ba8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:01:45.375803 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: WARNING py.warnings [None req-3c9a6cd0-b17d-4fa6-a096-99f5f9c8ba8c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 21:01:45.375803 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: warnings.warn(msg) Sep 09 21:01:45.375803 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: Sep 09 21:01:45.377044 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a6cd0-b17d-4fa6-a096-99f5f9c8ba8c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:01:45.512332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 750/1498] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:01:45 2022] POST /identity/v3/users => generated 287 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:01:45.516487 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd51a27-fc37-4642-a728-6e7b256205f8 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:01:45.518042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8fd51a27-fc37-4642-a728-6e7b256205f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:01:45.578294 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd51a27-fc37-4642-a728-6e7b256205f8 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:01:45.660767 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:01:45.671356 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0994ad-95af-42fd-a12f-8c774e3dcc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:01:45.671958 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0994ad-95af-42fd-a12f-8c774e3dcc18 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:01:45.672266 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0994ad-95af-42fd-a12f-8c774e3dcc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:01:45.672558 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0994ad-95af-42fd-a12f-8c774e3dcc18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:01:45.673192 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0994ad-95af-42fd-a12f-8c774e3dcc18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:01:45.674205 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: WARNING py.warnings [None req-fb0994ad-95af-42fd-a12f-8c774e3dcc18 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 21:01:45.674205 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: warnings.warn(msg) Sep 09 21:01:45.674205 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: Sep 09 21:01:45.675088 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0994ad-95af-42fd-a12f-8c774e3dcc18 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:01:45.689164 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 749/1499] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:01:45.692949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a6cd0-b17d-4fa6-a096-99f5f9c8ba8c admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:01:45.694538 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3c9a6cd0-b17d-4fa6-a096-99f5f9c8ba8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:01:45.706464 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a6cd0-b17d-4fa6-a096-99f5f9c8ba8c admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:01:45.732061 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:01:45.805328 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458cc49-9993-4767-97ec-c1aa5cb44a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:01:45.805942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458cc49-9993-4767-97ec-c1aa5cb44a97 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:01:45.806221 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458cc49-9993-4767-97ec-c1aa5cb44a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:01:45.806495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458cc49-9993-4767-97ec-c1aa5cb44a97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:01:45.807139 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7458cc49-9993-4767-97ec-c1aa5cb44a97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:01:45.808180 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: WARNING py.warnings [None req-7458cc49-9993-4767-97ec-c1aa5cb44a97 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 21:01:45.808180 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: warnings.warn(msg) Sep 09 21:01:45.808180 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: Sep 09 21:01:45.809072 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7458cc49-9993-4767-97ec-c1aa5cb44a97 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:01:45.821374 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 751/1500] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:01:45.825592 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0994ad-95af-42fd-a12f-8c774e3dcc18 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:01:45.827116 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fb0994ad-95af-42fd-a12f-8c774e3dcc18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:01:45.863029 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0994ad-95af-42fd-a12f-8c774e3dcc18 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:01:45.954450 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d598c-7b26-4761-8228-95de4279f8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:01:45.955107 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d598c-7b26-4761-8228-95de4279f8df admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:01:45.955382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d598c-7b26-4761-8228-95de4279f8df admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:01:45.955646 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d598c-7b26-4761-8228-95de4279f8df admin admin] PATH_INFO: `/v3/projects/d300b0e4f8534c68818a2aec3a94ed20/users/0d4164be05974656a3837540f35d9406/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:01:45.978826 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d598c-7b26-4761-8228-95de4279f8df admin admin] RBAC: Authorizing `identity:create_grant(project_id=d300b0e4f8534c68818a2aec3a94ed20, user_id=0d4164be05974656a3837540f35d9406, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:01:45.979876 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: WARNING py.warnings [None req-ee9d598c-7b26-4761-8228-95de4279f8df admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 21:01:45.979876 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: warnings.warn(msg) Sep 09 21:01:45.979876 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: Sep 09 21:01:45.982056 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d598c-7b26-4761-8228-95de4279f8df admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:01:46.093151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 750/1501] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:01:45 2022] PUT /identity/v3/projects/d300b0e4f8534c68818a2aec3a94ed20/users/0d4164be05974656a3837540f35d9406/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:01:46.105622 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668b715-a13f-4edd-a893-4b1de0acf619 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:01:46.105912 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668b715-a13f-4edd-a893-4b1de0acf619 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:01:46.106195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668b715-a13f-4edd-a893-4b1de0acf619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:01:46.162682 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-f668b715-a13f-4edd-a893-4b1de0acf619 None None] MFA Rules not processed for user `0d4164be05974656a3837540f35d9406`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:01:46.163897 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f668b715-a13f-4edd-a893-4b1de0acf619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:01:46.239938 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 752/1502] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:01:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:01:46.253125 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd1ad0-5fa8-4aad-86b9-7c52ef69ae67 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:01:46.253432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd1ad0-5fa8-4aad-86b9-7c52ef69ae67 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:01:46.253702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd1ad0-5fa8-4aad-86b9-7c52ef69ae67 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:01:46.255056 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 751/1503] 10.222.0.48 () {54 vars in 964 bytes} [Fri Sep 9 21:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:01:46.260324 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c379a0a2-22c6-4b49-9493-adc035d0d10b None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:01:46.260616 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c379a0a2-22c6-4b49-9493-adc035d0d10b None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:01:46.260892 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c379a0a2-22c6-4b49-9493-adc035d0d10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:01:46.293492 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:01:46.336493 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-c379a0a2-22c6-4b49-9493-adc035d0d10b None None] MFA Rules not processed for user `cf91131ee5bb481bbb47097cad0c003a`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:01:46.337696 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c379a0a2-22c6-4b49-9493-adc035d0d10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:01:46.411536 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 753/1504] 10.222.0.48 () {58 vars in 1061 bytes} [Fri Sep 9 21:01:46 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:01:46.419892 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd1ad0-5fa8-4aad-86b9-7c52ef69ae67 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:01:46.421505 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-abfd1ad0-5fa8-4aad-86b9-7c52ef69ae67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:01:46.470668 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:01:46.522462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd1ad0-5fa8-4aad-86b9-7c52ef69ae67 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:01:46.594212 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6053ea85-9067-40ca-80c9-c258a28b2621 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:01:46.594809 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053ea85-9067-40ca-80c9-c258a28b2621 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:01:46.595083 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053ea85-9067-40ca-80c9-c258a28b2621 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:01:46.595347 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053ea85-9067-40ca-80c9-c258a28b2621 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:01:46.596939 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6053ea85-9067-40ca-80c9-c258a28b2621 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:01:46.693379 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6053ea85-9067-40ca-80c9-c258a28b2621 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:01:46.694814 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6053ea85-9067-40ca-80c9-c258a28b2621 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:01:46.698535 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:01:46.713087 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 752/1505] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:01:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:01:46.757530 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9f643-3e94-4f50-90c8-e5045f959ade None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:01:46.757953 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9f643-3e94-4f50-90c8-e5045f959ade None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:01:46.758092 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9f643-3e94-4f50-90c8-e5045f959ade None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:01:46.759843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 754/1506] 10.222.0.48 () {52 vars in 830 bytes} [Fri Sep 9 21:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:01:46.763389 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6053ea85-9067-40ca-80c9-c258a28b2621 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:01:46.764639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6053ea85-9067-40ca-80c9-c258a28b2621 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:01:46.861666 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6053ea85-9067-40ca-80c9-c258a28b2621 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:01:46.954288 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca02c5a-52b3-4f6a-8105-400d7af4c771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:01:46.954916 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca02c5a-52b3-4f6a-8105-400d7af4c771 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:01:46.955194 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca02c5a-52b3-4f6a-8105-400d7af4c771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:01:46.955467 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca02c5a-52b3-4f6a-8105-400d7af4c771 admin admin] PATH_INFO: `/v3/users/0d4164be05974656a3837540f35d9406` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:01:46.968812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca02c5a-52b3-4f6a-8105-400d7af4c771 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d4164be05974656a3837540f35d9406)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:01:46.969842 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: WARNING py.warnings [None req-5ca02c5a-52b3-4f6a-8105-400d7af4c771 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 21:01:46.969842 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: warnings.warn(msg) Sep 09 21:01:46.969842 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: Sep 09 21:01:46.971051 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca02c5a-52b3-4f6a-8105-400d7af4c771 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:01:46.980800 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:01:47.193109 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5ca02c5a-52b3-4f6a-8105-400d7af4c771 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d4164be05974656a3837540f35d9406'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:01:47.193546 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5ca02c5a-52b3-4f6a-8105-400d7af4c771 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d4164be05974656a3837540f35d9406'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:01:47.194063 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5ca02c5a-52b3-4f6a-8105-400d7af4c771 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d4164be05974656a3837540f35d9406'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:01:47.208142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5ca02c5a-52b3-4f6a-8105-400d7af4c771 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d4164be05974656a3837540f35d9406'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:01:47.230938 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5ca02c5a-52b3-4f6a-8105-400d7af4c771 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d4164be05974656a3837540f35d9406'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:01:47.296647 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 753/1507] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:01:46 2022] DELETE /identity/v3/users/0d4164be05974656a3837540f35d9406 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:01:47.300953 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d149fb3-8c95-489b-aaf7-6072a3256a05 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:01:47.306119 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d149fb3-8c95-489b-aaf7-6072a3256a05 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:01:47.306119 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d149fb3-8c95-489b-aaf7-6072a3256a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:01:47.366249 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-8d149fb3-8c95-489b-aaf7-6072a3256a05 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:01:47.368773 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8d149fb3-8c95-489b-aaf7-6072a3256a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:01:47.443013 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:01:47.456615 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 755/1508] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:01:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:01:47.465092 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca02c5a-52b3-4f6a-8105-400d7af4c771 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:01:47.466390 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5ca02c5a-52b3-4f6a-8105-400d7af4c771 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:01:47.567465 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca02c5a-52b3-4f6a-8105-400d7af4c771 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:01:47.621299 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:01:47.640023 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e2c0d-0c8f-4eec-a7cb-bc204589af18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:01:47.640659 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e2c0d-0c8f-4eec-a7cb-bc204589af18 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:01:47.645886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e2c0d-0c8f-4eec-a7cb-bc204589af18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:01:47.645886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e2c0d-0c8f-4eec-a7cb-bc204589af18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:01:47.645886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2c3e2c0d-0c8f-4eec-a7cb-bc204589af18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:01:47.753008 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e2c0d-0c8f-4eec-a7cb-bc204589af18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:01:47.754386 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e2c0d-0c8f-4eec-a7cb-bc204589af18 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:01:47.772679 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 754/1509] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:01:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:01:48.225051 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d149fb3-8c95-489b-aaf7-6072a3256a05 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:01:48.226907 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8d149fb3-8c95-489b-aaf7-6072a3256a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:01:48.239948 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d149fb3-8c95-489b-aaf7-6072a3256a05 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:01:48.335190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30dcfd-9173-41b0-b9ef-54f180238ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:01:48.335814 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30dcfd-9173-41b0-b9ef-54f180238ab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:01:48.336115 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30dcfd-9173-41b0-b9ef-54f180238ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:01:48.337022 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30dcfd-9173-41b0-b9ef-54f180238ab2 admin admin] PATH_INFO: `/v3/projects/d300b0e4f8534c68818a2aec3a94ed20` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:01:48.346668 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a30dcfd-9173-41b0-b9ef-54f180238ab2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d300b0e4f8534c68818a2aec3a94ed20)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:01:48.347680 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: WARNING py.warnings [None req-0a30dcfd-9173-41b0-b9ef-54f180238ab2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 21:01:48.347680 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: warnings.warn(msg) Sep 09 21:01:48.347680 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: Sep 09 21:01:48.348898 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a30dcfd-9173-41b0-b9ef-54f180238ab2 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:01:48.483902 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0a30dcfd-9173-41b0-b9ef-54f180238ab2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd300b0e4f8534c68818a2aec3a94ed20', 'request_id': 'req-0a30dcfd-9173-41b0-b9ef-54f180238ab2'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:01:48.563220 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0a30dcfd-9173-41b0-b9ef-54f180238ab2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd300b0e4f8534c68818a2aec3a94ed20', 'request_id': 'req-0a30dcfd-9173-41b0-b9ef-54f180238ab2'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:01:48.579005 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0a30dcfd-9173-41b0-b9ef-54f180238ab2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd300b0e4f8534c68818a2aec3a94ed20', 'request_id': 'req-0a30dcfd-9173-41b0-b9ef-54f180238ab2'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:01:48.604856 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0a30dcfd-9173-41b0-b9ef-54f180238ab2 admin admin] The token cache is being invalidate because project d300b0e4f8534c68818a2aec3a94ed20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:01:48.608858 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0a30dcfd-9173-41b0-b9ef-54f180238ab2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:01:48.608858 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0a30dcfd-9173-41b0-b9ef-54f180238ab2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:01:48.608858 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 756/1510] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:01:48 2022] DELETE /identity/v3/projects/d300b0e4f8534c68818a2aec3a94ed20 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:01:57.770369 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=62 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:01:58.602772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:03:18.121789 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998716ad-c1b6-42f6-9959-e45abf294663 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:03:18.122384 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998716ad-c1b6-42f6-9959-e45abf294663 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:03:18.122384 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998716ad-c1b6-42f6-9959-e45abf294663 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:03:18.123666 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 21:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:03:18.128990 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb6b05-5cc0-4b38-b852-d2e7bd0338b0 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:03:18.129443 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb6b05-5cc0-4b38-b852-d2e7bd0338b0 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:03:18.129548 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb6b05-5cc0-4b38-b852-d2e7bd0338b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:03:18.209810 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-46fb6b05-5cc0-4b38-b852-d2e7bd0338b0 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:03:18.211024 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-46fb6b05-5cc0-4b38-b852-d2e7bd0338b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:03:18.313173 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 21:03:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:03:18.464827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998716ad-c1b6-42f6-9959-e45abf294663 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:03:18.467109 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-998716ad-c1b6-42f6-9959-e45abf294663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:03:18.584329 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998716ad-c1b6-42f6-9959-e45abf294663 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:03:18.590118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79c63a-6bc9-478c-bc56-e842c6c122ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:03:18.590716 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79c63a-6bc9-478c-bc56-e842c6c122ac service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:03:18.590993 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79c63a-6bc9-478c-bc56-e842c6c122ac service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:03:18.591260 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79c63a-6bc9-478c-bc56-e842c6c122ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:03:18.599109 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d79c63a-6bc9-478c-bc56-e842c6c122ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:03:18.600845 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d79c63a-6bc9-478c-bc56-e842c6c122ac service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:03:18.610639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 756/1513] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:03:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:03:19.681951 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d4313-d774-461c-b9dd-624659585e31 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:03:19.682541 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d4313-d774-461c-b9dd-624659585e31 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:03:19.682541 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d4313-d774-461c-b9dd-624659585e31 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:03:19.683829 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 758/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 21:03:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:03:19.690059 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5d242-f2bb-401e-a3e1-ae2aa6618a97 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:03:19.690508 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5d242-f2bb-401e-a3e1-ae2aa6618a97 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:03:19.690617 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5d242-f2bb-401e-a3e1-ae2aa6618a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:03:19.710842 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-43f5d242-f2bb-401e-a3e1-ae2aa6618a97 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:03:19.712732 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-43f5d242-f2bb-401e-a3e1-ae2aa6618a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:03:19.720465 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 757/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 21:03:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:03:19.872812 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d4313-d774-461c-b9dd-624659585e31 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:03:19.887905 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d4313-d774-461c-b9dd-624659585e31 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:03:19.893937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899acd57-7328-4cfe-be98-d073f36238e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:03:19.894534 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899acd57-7328-4cfe-be98-d073f36238e3 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:03:19.894658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59464364-bda5-492a-98ce-a853bea0bc0d None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:03:19.894813 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899acd57-7328-4cfe-be98-d073f36238e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:03:19.894919 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59464364-bda5-492a-98ce-a853bea0bc0d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:03:19.895085 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899acd57-7328-4cfe-be98-d073f36238e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:03:19.895187 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59464364-bda5-492a-98ce-a853bea0bc0d None None] PATH_INFO: `/v3` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:03:19.896462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 758/1516] 10.222.0.44 () {54 vars in 925 bytes} [Fri Sep 9 21:03:19 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:03:19.903598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b6054-5fd0-41e0-8c02-dffcb633fa7a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:03:19.903945 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b6054-5fd0-41e0-8c02-dffcb633fa7a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:03:19.904329 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b6054-5fd0-41e0-8c02-dffcb633fa7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:03:19.904437 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899acd57-7328-4cfe-be98-d073f36238e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:03:19.906586 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899acd57-7328-4cfe-be98-d073f36238e3 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:03:19.916296 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 759/1517] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:03:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:03:19.966500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-da8b6054-5fd0-41e0-8c02-dffcb633fa7a None None] MFA Rules not processed for user `2a4f6de2065d4fbea3fa141da760f708`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:03:19.967824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-da8b6054-5fd0-41e0-8c02-dffcb633fa7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:03:20.031831 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:03:20.050813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 759/1518] 10.222.0.44 () {58 vars in 1015 bytes} [Fri Sep 9 21:03:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:03:29.119493 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9f80f-f01a-467f-8db7-06cc58afe160 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:03:29.119941 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9f80f-f01a-467f-8db7-06cc58afe160 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:03:29.120186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9f80f-f01a-467f-8db7-06cc58afe160 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:03:29.121621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 760/1519] 10.222.0.48 () {54 vars in 961 bytes} [Fri Sep 9 21:03:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:03:29.126944 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ebb0b-0e27-4ac6-aac0-12ac540bb56d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:03:29.127393 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ebb0b-0e27-4ac6-aac0-12ac540bb56d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:03:29.127502 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ebb0b-0e27-4ac6-aac0-12ac540bb56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:03:29.147595 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ff0ebb0b-0e27-4ac6-aac0-12ac540bb56d None None] MFA Rules not processed for user `2a4f6de2065d4fbea3fa141da760f708`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:03:29.148808 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ff0ebb0b-0e27-4ac6-aac0-12ac540bb56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:03:29.157916 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 760/1520] 10.222.0.48 () {58 vars in 1058 bytes} [Fri Sep 9 21:03:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:03:29.166124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef9f80f-f01a-467f-8db7-06cc58afe160 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:03:29.179691 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef9f80f-f01a-467f-8db7-06cc58afe160 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:03:29.184671 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17edd2fa-cfae-493c-b34e-5bb1728cc711 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4f6de2065d4fbea3fa141da760f708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:03:29.185259 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17edd2fa-cfae-493c-b34e-5bb1728cc711 service placement] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:03:29.185534 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17edd2fa-cfae-493c-b34e-5bb1728cc711 service placement] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:03:29.185805 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17edd2fa-cfae-493c-b34e-5bb1728cc711 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:03:29.194123 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17edd2fa-cfae-493c-b34e-5bb1728cc711 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:03:29.195777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17edd2fa-cfae-493c-b34e-5bb1728cc711 service placement] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:03:29.205641 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 761/1521] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:03:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:03:31.381060 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047095a-07bc-472d-8d93-e36b94a2b37b None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:03:31.381679 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047095a-07bc-472d-8d93-e36b94a2b37b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:03:31.381679 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047095a-07bc-472d-8d93-e36b94a2b37b None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:03:31.382938 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 21:03:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:03:31.388920 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb2fe6-339f-447d-9ed2-5ad542cc4a46 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:03:31.389388 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb2fe6-339f-447d-9ed2-5ad542cc4a46 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:03:31.389507 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb2fe6-339f-447d-9ed2-5ad542cc4a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:03:31.409404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-8bdb2fe6-339f-447d-9ed2-5ad542cc4a46 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:03:31.410586 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8bdb2fe6-339f-447d-9ed2-5ad542cc4a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:03:31.418864 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 21:03:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:03:31.571846 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047095a-07bc-472d-8d93-e36b94a2b37b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:03:31.586352 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047095a-07bc-472d-8d93-e36b94a2b37b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:03:31.592474 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432ae3d-76fc-4997-a791-68858e0da579 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:03:31.593064 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432ae3d-76fc-4997-a791-68858e0da579 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:03:31.593343 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432ae3d-76fc-4997-a791-68858e0da579 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:03:31.593614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432ae3d-76fc-4997-a791-68858e0da579 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:03:31.601646 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4432ae3d-76fc-4997-a791-68858e0da579 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:03:31.603888 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4432ae3d-76fc-4997-a791-68858e0da579 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:03:31.613120 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 762/1524] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:03:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:03:32.673297 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d94d9-0c6c-4b7b-9656-8f0e55cac0da None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:03:32.673848 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d94d9-0c6c-4b7b-9656-8f0e55cac0da None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:03:32.673848 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d94d9-0c6c-4b7b-9656-8f0e55cac0da None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:03:32.675262 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 21:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:03:32.682012 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466ec7d-a7c4-438f-b3f3-5b45d9d56203 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:03:32.682453 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466ec7d-a7c4-438f-b3f3-5b45d9d56203 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:03:32.686039 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466ec7d-a7c4-438f-b3f3-5b45d9d56203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:03:32.702207 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-0466ec7d-a7c4-438f-b3f3-5b45d9d56203 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:03:32.703398 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0466ec7d-a7c4-438f-b3f3-5b45d9d56203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:03:32.711700 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 21:03:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:03:32.863088 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d94d9-0c6c-4b7b-9656-8f0e55cac0da None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:03:32.877133 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d94d9-0c6c-4b7b-9656-8f0e55cac0da None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:03:32.882425 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5abffa-ca20-4594-8b95-539e90674b3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:03:32.883007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5abffa-ca20-4594-8b95-539e90674b3b service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:03:32.883281 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5abffa-ca20-4594-8b95-539e90674b3b service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:03:32.883549 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5abffa-ca20-4594-8b95-539e90674b3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:03:32.891855 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5abffa-ca20-4594-8b95-539e90674b3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:03:32.893466 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5abffa-ca20-4594-8b95-539e90674b3b service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:03:32.903322 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 764/1527] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:03:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:03:37.178737 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b030425-71fc-4b84-a56d-858f2ac97163 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:03:37.179274 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b030425-71fc-4b84-a56d-858f2ac97163 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:03:37.179274 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b030425-71fc-4b84-a56d-858f2ac97163 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:03:37.181396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 21:03:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:03:37.186510 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db535575-2504-4535-8970-6ed2d0dc498a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:03:37.186929 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db535575-2504-4535-8970-6ed2d0dc498a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:03:37.187048 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db535575-2504-4535-8970-6ed2d0dc498a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:03:37.206482 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-db535575-2504-4535-8970-6ed2d0dc498a None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:03:37.207671 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-db535575-2504-4535-8970-6ed2d0dc498a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:03:37.215388 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 21:03:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:03:37.501845 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b030425-71fc-4b84-a56d-858f2ac97163 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:03:37.503675 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6b030425-71fc-4b84-a56d-858f2ac97163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:03:37.580898 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b030425-71fc-4b84-a56d-858f2ac97163 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:03:37.584449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0197a99-5f27-41cb-bace-ac1eb7397a57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:03:37.585042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0197a99-5f27-41cb-bace-ac1eb7397a57 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:03:37.585316 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0197a99-5f27-41cb-bace-ac1eb7397a57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:03:37.585592 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0197a99-5f27-41cb-bace-ac1eb7397a57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:03:37.594353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0197a99-5f27-41cb-bace-ac1eb7397a57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:03:37.595770 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0197a99-5f27-41cb-bace-ac1eb7397a57 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:03:37.604897 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 765/1530] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:03:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:03:39.755669 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f5340-e885-4ba9-9b20-d73d418fa1a9 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:03:39.756269 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f5340-e885-4ba9-9b20-d73d418fa1a9 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:03:39.756269 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f5340-e885-4ba9-9b20-d73d418fa1a9 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:03:39.757903 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 21:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:03:39.763461 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20f031-594f-47bc-b65b-ac196db1b80f None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:03:39.763988 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20f031-594f-47bc-b65b-ac196db1b80f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:03:39.763988 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20f031-594f-47bc-b65b-ac196db1b80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:03:39.787079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1c20f031-594f-47bc-b65b-ac196db1b80f None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:03:39.788870 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1c20f031-594f-47bc-b65b-ac196db1b80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:03:39.801070 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 21:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:03:40.086172 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f5340-e885-4ba9-9b20-d73d418fa1a9 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:03:40.092266 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=94 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:03:40.097969 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f5340-e885-4ba9-9b20-d73d418fa1a9 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:03:40.101463 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760eeb6-7f0d-43fd-9319-97d43b17fdaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:03:40.102076 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760eeb6-7f0d-43fd-9319-97d43b17fdaa service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:03:40.102346 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760eeb6-7f0d-43fd-9319-97d43b17fdaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:03:40.102613 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760eeb6-7f0d-43fd-9319-97d43b17fdaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:03:40.111811 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1760eeb6-7f0d-43fd-9319-97d43b17fdaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:03:40.113262 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1760eeb6-7f0d-43fd-9319-97d43b17fdaa service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:03:40.122551 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 767/1533] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:03:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:03:44.002264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647d844-27f0-4733-a5c6-20b8a0967501 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:03:44.002799 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647d844-27f0-4733-a5c6-20b8a0967501 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:03:44.002799 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647d844-27f0-4733-a5c6-20b8a0967501 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:03:44.004062 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 21:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:03:44.009857 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca29b92d-86ce-4a31-aa25-c29e5d2d65f7 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:03:44.010279 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca29b92d-86ce-4a31-aa25-c29e5d2d65f7 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:03:44.010482 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca29b92d-86ce-4a31-aa25-c29e5d2d65f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:03:44.030399 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-ca29b92d-86ce-4a31-aa25-c29e5d2d65f7 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:03:44.031596 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ca29b92d-86ce-4a31-aa25-c29e5d2d65f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:03:44.039360 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 21:03:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:03:44.339692 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647d844-27f0-4733-a5c6-20b8a0967501 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:03:44.341384 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5647d844-27f0-4733-a5c6-20b8a0967501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:03:44.418496 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647d844-27f0-4733-a5c6-20b8a0967501 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:03:44.422064 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68865df8-1a4c-4b41-bd50-f881400d0d6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:03:44.422657 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68865df8-1a4c-4b41-bd50-f881400d0d6b service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:03:44.422935 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68865df8-1a4c-4b41-bd50-f881400d0d6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:03:44.423205 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68865df8-1a4c-4b41-bd50-f881400d0d6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:03:44.432551 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68865df8-1a4c-4b41-bd50-f881400d0d6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:03:44.433980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68865df8-1a4c-4b41-bd50-f881400d0d6b service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:03:44.443155 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 768/1536] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:03:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:03:46.563020 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9cbb7-897d-4955-97f8-3e66f5ce8764 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:03:46.563595 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9cbb7-897d-4955-97f8-3e66f5ce8764 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:03:46.563595 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9cbb7-897d-4955-97f8-3e66f5ce8764 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:03:46.564963 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 21:03:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:03:46.570300 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab9780-80d1-4eaf-a263-551e84d846bd None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:03:46.570725 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab9780-80d1-4eaf-a263-551e84d846bd None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:03:46.570852 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab9780-80d1-4eaf-a263-551e84d846bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:03:46.590766 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-cbab9780-80d1-4eaf-a263-551e84d846bd None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:03:46.591962 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cbab9780-80d1-4eaf-a263-551e84d846bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:03:46.599694 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 21:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:03:46.902868 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c9cbb7-897d-4955-97f8-3e66f5ce8764 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:03:46.904617 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-94c9cbb7-897d-4955-97f8-3e66f5ce8764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:03:46.917549 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c9cbb7-897d-4955-97f8-3e66f5ce8764 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:03:46.921096 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f9899-ae87-47ff-a27d-d1a7c4f1690d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:03:46.921680 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f9899-ae87-47ff-a27d-d1a7c4f1690d service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:03:46.921961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f9899-ae87-47ff-a27d-d1a7c4f1690d service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:03:46.922238 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f9899-ae87-47ff-a27d-d1a7c4f1690d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:03:46.931034 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f9899-ae87-47ff-a27d-d1a7c4f1690d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:03:46.932519 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f9899-ae87-47ff-a27d-d1a7c4f1690d service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:03:46.941736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 770/1539] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:03:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:03:56.585447 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:03:56.937660 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=14 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:12.700952 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd64b0fe-629e-4940-be51-0df9d94bb367 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:12.708763 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd64b0fe-629e-4940-be51-0df9d94bb367 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:12.708763 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd64b0fe-629e-4940-be51-0df9d94bb367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:12.733898 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-cd64b0fe-629e-4940-be51-0df9d94bb367 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:12.735599 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cd64b0fe-629e-4940-be51-0df9d94bb367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:12.744900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 770/1540] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:05:12.753822 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5d2e6-b98a-4279-84a1-7448a7512254 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:12.754840 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5d2e6-b98a-4279-84a1-7448a7512254 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:12.755428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5d2e6-b98a-4279-84a1-7448a7512254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:12.783406 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-cbb5d2e6-b98a-4279-84a1-7448a7512254 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:12.784981 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-cbb5d2e6-b98a-4279-84a1-7448a7512254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:12.799242 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 771/1541] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:12.804102 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd64b0fe-629e-4940-be51-0df9d94bb367 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:12.815355 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd64b0fe-629e-4940-be51-0df9d94bb367 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:12.825536 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fc6cc2-2985-4f99-980b-fe2b74c1767c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:12.826596 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc6cc2-2985-4f99-980b-fe2b74c1767c admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:12.827074 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc6cc2-2985-4f99-980b-fe2b74c1767c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:12.827554 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc6cc2-2985-4f99-980b-fe2b74c1767c admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:12.828454 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fc6cc2-2985-4f99-980b-fe2b74c1767c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:05:12.828964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fc6cc2-2985-4f99-980b-fe2b74c1767c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:12.830183 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: WARNING py.warnings [None req-57fc6cc2-2985-4f99-980b-fe2b74c1767c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 21:05:12.830183 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: warnings.warn(msg) Sep 09 21:05:12.830183 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: Sep 09 21:05:12.831463 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fc6cc2-2985-4f99-980b-fe2b74c1767c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:12.848328 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 771/1542] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:05:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:05:12.856704 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5d2e6-b98a-4279-84a1-7448a7512254 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:12.867089 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5d2e6-b98a-4279-84a1-7448a7512254 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:12.876958 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8734ba05-ce01-4be4-ba4c-3128ad810220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:12.877947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734ba05-ce01-4be4-ba4c-3128ad810220 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:12.878436 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734ba05-ce01-4be4-ba4c-3128ad810220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:12.878907 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734ba05-ce01-4be4-ba4c-3128ad810220 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:12.879780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8734ba05-ce01-4be4-ba4c-3128ad810220 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:12.881044 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: WARNING py.warnings [None req-8734ba05-ce01-4be4-ba4c-3128ad810220 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 21:05:12.881044 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: warnings.warn(msg) Sep 09 21:05:12.881044 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: Sep 09 21:05:12.882609 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8734ba05-ce01-4be4-ba4c-3128ad810220 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:13.020244 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb042972-f9ab-4938-8469-d183e6b007ce None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:13.021131 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb042972-f9ab-4938-8469-d183e6b007ce None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:13.021646 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb042972-f9ab-4938-8469-d183e6b007ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:13.054032 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 772/1543] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:05:12 2022] POST /identity/v3/projects => generated 391 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:13.055536 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-bb042972-f9ab-4938-8469-d183e6b007ce None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:13.057530 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bb042972-f9ab-4938-8469-d183e6b007ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:13.058319 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8734ba05-ce01-4be4-ba4c-3128ad810220 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:13.183477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 772/1544] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:13.192118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ef56f-c79a-4971-87b9-4c470196fbbb None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:13.192118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ef56f-c79a-4971-87b9-4c470196fbbb None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:13.192118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ef56f-c79a-4971-87b9-4c470196fbbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:13.193478 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8734ba05-ce01-4be4-ba4c-3128ad810220 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:13.208629 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffcb90d-e64e-4b4f-bf0a-63eec6d4e2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:13.209611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcb90d-e64e-4b4f-bf0a-63eec6d4e2f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:13.210114 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcb90d-e64e-4b4f-bf0a-63eec6d4e2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:13.210606 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcb90d-e64e-4b4f-bf0a-63eec6d4e2f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:13.212167 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffcb90d-e64e-4b4f-bf0a-63eec6d4e2f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:13.214769 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffcb90d-e64e-4b4f-bf0a-63eec6d4e2f1 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:13.222276 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5c3ef56f-c79a-4971-87b9-4c470196fbbb None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:13.223583 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5c3ef56f-c79a-4971-87b9-4c470196fbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:13.233092 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 773/1545] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:13.233463 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454220aa-6ae0-41ea-9c7f-d734cafda9ef None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:13.233753 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454220aa-6ae0-41ea-9c7f-d734cafda9ef None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:13.234088 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454220aa-6ae0-41ea-9c7f-d734cafda9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:13.256033 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-454220aa-6ae0-41ea-9c7f-d734cafda9ef None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:13.260568 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-454220aa-6ae0-41ea-9c7f-d734cafda9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:13.266712 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 774/1546] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:13.267919 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1720e57-e381-4dff-be6d-7ed1e930146d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:13.268241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1720e57-e381-4dff-be6d-7ed1e930146d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:13.268672 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1720e57-e381-4dff-be6d-7ed1e930146d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:13.290787 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c1720e57-e381-4dff-be6d-7ed1e930146d None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:13.292051 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c1720e57-e381-4dff-be6d-7ed1e930146d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:13.301067 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 775/1547] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:13.302529 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e77a9c-9009-48ac-83f8-653594ad59bc None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:13.303104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e77a9c-9009-48ac-83f8-653594ad59bc None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:13.303535 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e77a9c-9009-48ac-83f8-653594ad59bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:13.316171 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 773/1548] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:05:13 2022] POST /identity/v3/users => generated 296 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:05:13.317708 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4bb389-1fff-4c42-bb77-bdf815289b0d None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:13.318074 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4bb389-1fff-4c42-bb77-bdf815289b0d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:13.318293 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4bb389-1fff-4c42-bb77-bdf815289b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:13.327038 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-48e77a9c-9009-48ac-83f8-653594ad59bc None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:13.328270 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-48e77a9c-9009-48ac-83f8-653594ad59bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:13.337027 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 776/1549] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:13.338232 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787a2a5-c5a2-4a67-8e0e-a826fba72b3e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:13.338511 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787a2a5-c5a2-4a67-8e0e-a826fba72b3e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:13.338777 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787a2a5-c5a2-4a67-8e0e-a826fba72b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:13.339069 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-4f4bb389-1fff-4c42-bb77-bdf815289b0d None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:13.340284 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-4f4bb389-1fff-4c42-bb77-bdf815289b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:13.348592 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 774/1550] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:13.349813 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b821a-1d81-43c3-8cd9-33a0bde98269 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:13.350091 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b821a-1d81-43c3-8cd9-33a0bde98269 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:13.350355 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b821a-1d81-43c3-8cd9-33a0bde98269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:13.358230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-0787a2a5-c5a2-4a67-8e0e-a826fba72b3e None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:13.359425 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0787a2a5-c5a2-4a67-8e0e-a826fba72b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:13.367696 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 777/1551] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:13.368884 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de9efcf-2374-4cb3-a2e8-95353dd2cf93 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:13.369161 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de9efcf-2374-4cb3-a2e8-95353dd2cf93 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:13.369430 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de9efcf-2374-4cb3-a2e8-95353dd2cf93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:13.370013 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-0a5b821a-1d81-43c3-8cd9-33a0bde98269 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:13.371241 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0a5b821a-1d81-43c3-8cd9-33a0bde98269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:13.379515 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 775/1552] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:13.380339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b821a-1d81-43c3-8cd9-33a0bde98269 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:13.390309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1de9efcf-2374-4cb3-a2e8-95353dd2cf93 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:13.390505 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b821a-1d81-43c3-8cd9-33a0bde98269 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:13.391485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1de9efcf-2374-4cb3-a2e8-95353dd2cf93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:13.394824 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7ce5b-3aa1-4e6a-bdec-dce3ebff724b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:13.395408 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7ce5b-3aa1-4e6a-bdec-dce3ebff724b admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:13.395684 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7ce5b-3aa1-4e6a-bdec-dce3ebff724b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:13.395951 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7ce5b-3aa1-4e6a-bdec-dce3ebff724b admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:13.396638 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7ce5b-3aa1-4e6a-bdec-dce3ebff724b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:05:13.399530 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7ce5b-3aa1-4e6a-bdec-dce3ebff724b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:13.399530 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7ce5b-3aa1-4e6a-bdec-dce3ebff724b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:13.399864 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 778/1553] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:13.404216 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da52b8-cf3c-4f81-95a9-d82ffec882cd None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:13.404216 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da52b8-cf3c-4f81-95a9-d82ffec882cd None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:13.404216 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da52b8-cf3c-4f81-95a9-d82ffec882cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:13.420253 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 776/1554] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:05:13.421187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7ce5b-3aa1-4e6a-bdec-dce3ebff724b admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:13.421584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-74da52b8-cf3c-4f81-95a9-d82ffec882cd None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:13.422714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-74da52b8-cf3c-4f81-95a9-d82ffec882cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:13.431174 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 779/1555] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:13.432320 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7ce5b-3aa1-4e6a-bdec-dce3ebff724b admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:13.432453 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a3809-56a6-4ac8-b7a1-8e43c81f6d06 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:13.432635 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a3809-56a6-4ac8-b7a1-8e43c81f6d06 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:13.432957 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a3809-56a6-4ac8-b7a1-8e43c81f6d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:13.437044 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a336596-da68-44be-b2f1-a32eec9cc224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:13.437639 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a336596-da68-44be-b2f1-a32eec9cc224 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:13.438210 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a336596-da68-44be-b2f1-a32eec9cc224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:13.438336 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a336596-da68-44be-b2f1-a32eec9cc224 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:13.438959 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a336596-da68-44be-b2f1-a32eec9cc224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:13.440580 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a336596-da68-44be-b2f1-a32eec9cc224 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:13.452135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 777/1556] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:13.452894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a18a3809-56a6-4ac8-b7a1-8e43c81f6d06 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:13.456696 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a336596-da68-44be-b2f1-a32eec9cc224 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:13.456805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a18a3809-56a6-4ac8-b7a1-8e43c81f6d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:13.462104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 780/1557] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:13.462690 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a336596-da68-44be-b2f1-a32eec9cc224 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:13.463310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad44280-71f7-403c-90f0-dc18cc4b078d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:13.463581 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad44280-71f7-403c-90f0-dc18cc4b078d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:13.463827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad44280-71f7-403c-90f0-dc18cc4b078d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:13.467737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba4edd-bdfb-4da0-bcc1-2411293349bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:13.468405 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba4edd-bdfb-4da0-bcc1-2411293349bb admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:13.468681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba4edd-bdfb-4da0-bcc1-2411293349bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:13.468951 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba4edd-bdfb-4da0-bcc1-2411293349bb admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:13.469583 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba4edd-bdfb-4da0-bcc1-2411293349bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:05:13.469879 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba4edd-bdfb-4da0-bcc1-2411293349bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:13.471441 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba4edd-bdfb-4da0-bcc1-2411293349bb admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:13.482728 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-dad44280-71f7-403c-90f0-dc18cc4b078d None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:13.482908 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 778/1558] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:05:13.483650 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba4edd-bdfb-4da0-bcc1-2411293349bb admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:13.484426 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-dad44280-71f7-403c-90f0-dc18cc4b078d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:13.493658 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba4edd-bdfb-4da0-bcc1-2411293349bb admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:13.502455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 781/1559] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:13.502455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb9c05-41ad-4d3c-86f2-a3c2c3b080af None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:13.502455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb9c05-41ad-4d3c-86f2-a3c2c3b080af None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:13.502455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb9c05-41ad-4d3c-86f2-a3c2c3b080af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:13.502858 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb275f34-ff20-49db-8dcd-0f153cbaeb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:13.502858 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb275f34-ff20-49db-8dcd-0f153cbaeb62 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:13.502858 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb275f34-ff20-49db-8dcd-0f153cbaeb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:13.502858 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb275f34-ff20-49db-8dcd-0f153cbaeb62 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:13.503190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb275f34-ff20-49db-8dcd-0f153cbaeb62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:05:13.503190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb275f34-ff20-49db-8dcd-0f153cbaeb62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:13.503190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb275f34-ff20-49db-8dcd-0f153cbaeb62 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:13.521939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 779/1560] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:05:13.521939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb275f34-ff20-49db-8dcd-0f153cbaeb62 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:13.522230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-42bb9c05-41ad-4d3c-86f2-a3c2c3b080af None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:13.522230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-42bb9c05-41ad-4d3c-86f2-a3c2c3b080af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:13.525238 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 782/1561] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:13.529630 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0d865-501c-4c55-8ba0-8a09f82dff1e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:13.529630 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0d865-501c-4c55-8ba0-8a09f82dff1e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:13.529630 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0d865-501c-4c55-8ba0-8a09f82dff1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:13.529897 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb275f34-ff20-49db-8dcd-0f153cbaeb62 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:13.534217 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ff075c-f1bf-419a-9ccc-15b469158421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:13.534802 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff075c-f1bf-419a-9ccc-15b469158421 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:13.535079 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff075c-f1bf-419a-9ccc-15b469158421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:13.535346 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff075c-f1bf-419a-9ccc-15b469158421 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:13.535997 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ff075c-f1bf-419a-9ccc-15b469158421 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:13.537908 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ff075c-f1bf-419a-9ccc-15b469158421 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:13.547140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a3b0d865-501c-4c55-8ba0-8a09f82dff1e None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:13.548350 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a3b0d865-501c-4c55-8ba0-8a09f82dff1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:13.556626 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 783/1562] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:13.557495 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b0d865-501c-4c55-8ba0-8a09f82dff1e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:13.566424 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b0d865-501c-4c55-8ba0-8a09f82dff1e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:13.571799 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f24a25-ec2b-4074-aa80-36c829696e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:13.572438 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f24a25-ec2b-4074-aa80-36c829696e1b admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:13.572722 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f24a25-ec2b-4074-aa80-36c829696e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:13.572998 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f24a25-ec2b-4074-aa80-36c829696e1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:13.573661 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f24a25-ec2b-4074-aa80-36c829696e1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:13.575651 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f24a25-ec2b-4074-aa80-36c829696e1b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:13.585467 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 784/1563] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:13.586432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f24a25-ec2b-4074-aa80-36c829696e1b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:13.596237 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f24a25-ec2b-4074-aa80-36c829696e1b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:13.600765 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11a3d3-fc41-4500-b58f-948c06399114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:13.601308 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11a3d3-fc41-4500-b58f-948c06399114 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:13.601659 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11a3d3-fc41-4500-b58f-948c06399114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:13.601937 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11a3d3-fc41-4500-b58f-948c06399114 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:13.602563 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11a3d3-fc41-4500-b58f-948c06399114 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:05:13.602866 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11a3d3-fc41-4500-b58f-948c06399114 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:13.604433 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11a3d3-fc41-4500-b58f-948c06399114 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:13.615964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 785/1564] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:05:13.616872 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11a3d3-fc41-4500-b58f-948c06399114 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:13.626988 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11a3d3-fc41-4500-b58f-948c06399114 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:13.631832 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d89a9-f83d-455e-a243-2c980c2b3f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:13.632470 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d89a9-f83d-455e-a243-2c980c2b3f10 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:13.636845 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d89a9-f83d-455e-a243-2c980c2b3f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:13.636845 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d89a9-f83d-455e-a243-2c980c2b3f10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:13.636845 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d89a9-f83d-455e-a243-2c980c2b3f10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:13.638043 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d89a9-f83d-455e-a243-2c980c2b3f10 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:13.642778 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 780/1565] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:05:13 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:13.644259 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ff075c-f1bf-419a-9ccc-15b469158421 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:13.678036 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:13.710054 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ff075c-f1bf-419a-9ccc-15b469158421 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:13.714717 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a03ac-d3f1-437b-bdeb-dc295caaf233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:13.717069 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a03ac-d3f1-437b-bdeb-dc295caaf233 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:13.717069 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a03ac-d3f1-437b-bdeb-dc295caaf233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:13.717069 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a03ac-d3f1-437b-bdeb-dc295caaf233 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:13.717069 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1a03ac-d3f1-437b-bdeb-dc295caaf233 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:05:13.717069 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1a03ac-d3f1-437b-bdeb-dc295caaf233 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:13.718416 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1a03ac-d3f1-437b-bdeb-dc295caaf233 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:13.729705 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 781/1566] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:05:13.730602 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a03ac-d3f1-437b-bdeb-dc295caaf233 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:13.739858 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a03ac-d3f1-437b-bdeb-dc295caaf233 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:13.745183 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 786/1567] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:05:13 2022] POST /identity/v3/projects => generated 389 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:13.749808 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac268e21-8efd-4d1e-8cac-67b99f6c974c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:13.749808 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac268e21-8efd-4d1e-8cac-67b99f6c974c admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:13.749808 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac268e21-8efd-4d1e-8cac-67b99f6c974c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:13.749808 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac268e21-8efd-4d1e-8cac-67b99f6c974c admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:13.750181 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac268e21-8efd-4d1e-8cac-67b99f6c974c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:13.750276 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d89a9-f83d-455e-a243-2c980c2b3f10 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:13.750366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac268e21-8efd-4d1e-8cac-67b99f6c974c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:13.776310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:13.809580 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d89a9-f83d-455e-a243-2c980c2b3f10 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:13.814228 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a4494-1e3c-4bd4-82a6-73c41cb22695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:13.814830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a4494-1e3c-4bd4-82a6-73c41cb22695 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:13.815111 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a4494-1e3c-4bd4-82a6-73c41cb22695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:13.815385 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a4494-1e3c-4bd4-82a6-73c41cb22695 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:13.816022 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a4494-1e3c-4bd4-82a6-73c41cb22695 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:05:13.816357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a4494-1e3c-4bd4-82a6-73c41cb22695 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:13.817950 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a4494-1e3c-4bd4-82a6-73c41cb22695 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:13.829169 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 787/1568] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:05:13.829943 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a4494-1e3c-4bd4-82a6-73c41cb22695 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:13.841061 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a4494-1e3c-4bd4-82a6-73c41cb22695 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:13.841455 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 782/1569] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:05:13 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:13.842427 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac268e21-8efd-4d1e-8cac-67b99f6c974c admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:13.899592 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ecc7e1-b281-4f3c-90df-6fca0494ccab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:13.900240 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ecc7e1-b281-4f3c-90df-6fca0494ccab admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:13.900527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ecc7e1-b281-4f3c-90df-6fca0494ccab admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:13.900800 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ecc7e1-b281-4f3c-90df-6fca0494ccab admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:13.901444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ecc7e1-b281-4f3c-90df-6fca0494ccab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:05:13.901751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ecc7e1-b281-4f3c-90df-6fca0494ccab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:13.903440 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ecc7e1-b281-4f3c-90df-6fca0494ccab admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:13.906619 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac268e21-8efd-4d1e-8cac-67b99f6c974c admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:13.911419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f877dc-51ca-45c8-95cb-c32d04b1b04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:13.912089 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f877dc-51ca-45c8-95cb-c32d04b1b04f admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:13.912395 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f877dc-51ca-45c8-95cb-c32d04b1b04f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:13.912670 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f877dc-51ca-45c8-95cb-c32d04b1b04f admin admin] PATH_INFO: `/v3/projects/25acf91bdf524474b03d44271d08fba6/users/c764d9778ef74b709cedbccabac37af0/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:13.916646 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 788/1570] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:05:13.917462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ecc7e1-b281-4f3c-90df-6fca0494ccab admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:13.928006 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ecc7e1-b281-4f3c-90df-6fca0494ccab admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:13.933041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f877dc-51ca-45c8-95cb-c32d04b1b04f admin admin] RBAC: Authorizing `identity:create_grant(project_id=25acf91bdf524474b03d44271d08fba6, user_id=c764d9778ef74b709cedbccabac37af0, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:13.933246 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a12d38-e5e7-44a3-9694-861d069e3f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:13.933863 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a12d38-e5e7-44a3-9694-861d069e3f69 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:13.934157 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a12d38-e5e7-44a3-9694-861d069e3f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:13.934433 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a12d38-e5e7-44a3-9694-861d069e3f69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:13.934544 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: WARNING py.warnings [None req-b1f877dc-51ca-45c8-95cb-c32d04b1b04f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 21:05:13.934544 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: warnings.warn(msg) Sep 09 21:05:13.934544 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: Sep 09 21:05:13.935061 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a12d38-e5e7-44a3-9694-861d069e3f69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:13.936593 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f877dc-51ca-45c8-95cb-c32d04b1b04f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:13.936995 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a12d38-e5e7-44a3-9694-861d069e3f69 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:14.011648 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 783/1571] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:13 2022] PUT /identity/v3/projects/25acf91bdf524474b03d44271d08fba6/users/c764d9778ef74b709cedbccabac37af0/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:14.012316 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f877dc-51ca-45c8-95cb-c32d04b1b04f admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:14.044776 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 789/1572] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:05:13 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:14.044776 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a12d38-e5e7-44a3-9694-861d069e3f69 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:14.074721 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f877dc-51ca-45c8-95cb-c32d04b1b04f admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:14.103066 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:14.106202 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a12d38-e5e7-44a3-9694-861d069e3f69 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:14.110760 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15ade8-ba96-4ee2-a0d5-78240b3e9fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:14.112526 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15ade8-ba96-4ee2-a0d5-78240b3e9fb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:14.112526 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15ade8-ba96-4ee2-a0d5-78240b3e9fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:14.112526 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15ade8-ba96-4ee2-a0d5-78240b3e9fb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:14.112827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15ade8-ba96-4ee2-a0d5-78240b3e9fb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:14.114653 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15ade8-ba96-4ee2-a0d5-78240b3e9fb0 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:14.132696 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8a326-d925-4f3c-bfb0-1f61f06c677e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:14.133311 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8a326-d925-4f3c-bfb0-1f61f06c677e admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:14.133584 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8a326-d925-4f3c-bfb0-1f61f06c677e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:14.133856 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8a326-d925-4f3c-bfb0-1f61f06c677e admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:14.134489 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb8a326-d925-4f3c-bfb0-1f61f06c677e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:14.136451 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb8a326-d925-4f3c-bfb0-1f61f06c677e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:14.191457 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 790/1573] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:05:14 2022] POST /identity/v3/projects => generated 375 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:14.192306 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15ade8-ba96-4ee2-a0d5-78240b3e9fb0 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:14.205347 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:14.217448 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 784/1574] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:05:14 2022] POST /identity/v3/users => generated 287 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:14.217448 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8a326-d925-4f3c-bfb0-1f61f06c677e admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:14.224938 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8a326-d925-4f3c-bfb0-1f61f06c677e admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:14.254305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15ade8-ba96-4ee2-a0d5-78240b3e9fb0 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:14.259263 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b188ff4-4f79-4f51-8925-18edb6fc88fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:14.259856 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b188ff4-4f79-4f51-8925-18edb6fc88fb admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:14.260179 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b188ff4-4f79-4f51-8925-18edb6fc88fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:14.260440 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b188ff4-4f79-4f51-8925-18edb6fc88fb admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:14.261127 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b188ff4-4f79-4f51-8925-18edb6fc88fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:14.262162 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: WARNING py.warnings [None req-4b188ff4-4f79-4f51-8925-18edb6fc88fb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 21:05:14.262162 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: warnings.warn(msg) Sep 09 21:05:14.262162 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: Sep 09 21:05:14.263814 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b188ff4-4f79-4f51-8925-18edb6fc88fb admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:14.291503 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b8c362-fe61-4db0-b16d-43db62f4be09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:14.292147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8c362-fe61-4db0-b16d-43db62f4be09 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:14.292425 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8c362-fe61-4db0-b16d-43db62f4be09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:14.292692 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8c362-fe61-4db0-b16d-43db62f4be09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:14.293329 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b8c362-fe61-4db0-b16d-43db62f4be09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:14.295733 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b8c362-fe61-4db0-b16d-43db62f4be09 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:14.393793 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 791/1575] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:05:14 2022] POST /identity/v3/users => generated 295 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:14.394751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b188ff4-4f79-4f51-8925-18edb6fc88fb admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:14.404141 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b188ff4-4f79-4f51-8925-18edb6fc88fb admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:14.408679 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e813ab3-63db-49a9-94c2-4bfe1028a9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:14.409376 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e813ab3-63db-49a9-94c2-4bfe1028a9c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:14.409527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e813ab3-63db-49a9-94c2-4bfe1028a9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:14.409799 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e813ab3-63db-49a9-94c2-4bfe1028a9c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:14.410412 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e813ab3-63db-49a9-94c2-4bfe1028a9c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:14.412363 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e813ab3-63db-49a9-94c2-4bfe1028a9c5 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:14.421104 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 785/1576] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:05:14 2022] POST /identity/v3/projects => generated 375 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:14.421104 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b8c362-fe61-4db0-b16d-43db62f4be09 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:14.482291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b8c362-fe61-4db0-b16d-43db62f4be09 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:14.487031 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c0bd8-b5dd-434e-b3ea-8bf715f8a598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:14.487627 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c0bd8-b5dd-434e-b3ea-8bf715f8a598 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:14.487908 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c0bd8-b5dd-434e-b3ea-8bf715f8a598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:14.488211 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c0bd8-b5dd-434e-b3ea-8bf715f8a598 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:14.488840 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c0bd8-b5dd-434e-b3ea-8bf715f8a598 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:14.490731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c0bd8-b5dd-434e-b3ea-8bf715f8a598 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:14.538732 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 792/1577] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:05:14 2022] POST /identity/v3/users => generated 289 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:14.540553 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e2563-b911-46af-a09e-6f067f01d7c5 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:14.540971 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e2563-b911-46af-a09e-6f067f01d7c5 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:14.541253 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e2563-b911-46af-a09e-6f067f01d7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:14.560561 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-465e2563-b911-46af-a09e-6f067f01d7c5 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:14.561749 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-465e2563-b911-46af-a09e-6f067f01d7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:14.566799 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 786/1578] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:05:14 2022] POST /identity/v3/projects => generated 365 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:14.572412 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c0bd8-b5dd-434e-b3ea-8bf715f8a598 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:14.572518 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 793/1579] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:14.572518 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e2563-b911-46af-a09e-6f067f01d7c5 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:14.584656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e2563-b911-46af-a09e-6f067f01d7c5 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:14.605868 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:14.639452 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c0bd8-b5dd-434e-b3ea-8bf715f8a598 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:14.644188 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eb0f1a-2ad4-4941-93de-7f785e40bd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:14.644801 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb0f1a-2ad4-4941-93de-7f785e40bd2f admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:14.645079 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb0f1a-2ad4-4941-93de-7f785e40bd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:14.645338 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb0f1a-2ad4-4941-93de-7f785e40bd2f admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:14.645981 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eb0f1a-2ad4-4941-93de-7f785e40bd2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:14.647900 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eb0f1a-2ad4-4941-93de-7f785e40bd2f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:14.651806 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9becb-c52f-4c18-804e-aa7f400d6f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:14.652483 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9becb-c52f-4c18-804e-aa7f400d6f3e admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:14.652792 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9becb-c52f-4c18-804e-aa7f400d6f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:14.653085 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9becb-c52f-4c18-804e-aa7f400d6f3e admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:14.653747 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe9becb-c52f-4c18-804e-aa7f400d6f3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:14.655769 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe9becb-c52f-4c18-804e-aa7f400d6f3e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:14.731237 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 787/1580] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:05:14 2022] POST /identity/v3/users => generated 287 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:14.732126 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eb0f1a-2ad4-4941-93de-7f785e40bd2f admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:14.741419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eb0f1a-2ad4-4941-93de-7f785e40bd2f admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:14.746534 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7655235-a895-4f65-8c9e-da5c8f420eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:14.747227 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7655235-a895-4f65-8c9e-da5c8f420eab admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:14.747507 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7655235-a895-4f65-8c9e-da5c8f420eab admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:14.747787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7655235-a895-4f65-8c9e-da5c8f420eab admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:14.748458 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7655235-a895-4f65-8c9e-da5c8f420eab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:14.750031 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7655235-a895-4f65-8c9e-da5c8f420eab admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:14.760182 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 794/1581] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:05:14 2022] POST /identity/v3/users => generated 288 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:14.763300 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9becb-c52f-4c18-804e-aa7f400d6f3e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:14.763508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 788/1582] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:14.763508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7655235-a895-4f65-8c9e-da5c8f420eab admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:14.771410 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9becb-c52f-4c18-804e-aa7f400d6f3e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:14.772655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7655235-a895-4f65-8c9e-da5c8f420eab admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:14.776226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c4d20-7f2c-41f8-8778-ee6023099b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:14.776849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c4d20-7f2c-41f8-8778-ee6023099b28 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:14.777130 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c4d20-7f2c-41f8-8778-ee6023099b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:14.777279 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073571a-fba3-4ddc-ba3b-21c5d6e70c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:14.777422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c4d20-7f2c-41f8-8778-ee6023099b28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:14.777862 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073571a-fba3-4ddc-ba3b-21c5d6e70c53 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:14.778036 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c4d20-7f2c-41f8-8778-ee6023099b28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:14.778145 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073571a-fba3-4ddc-ba3b-21c5d6e70c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:14.778403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073571a-fba3-4ddc-ba3b-21c5d6e70c53 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:14.779026 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d073571a-fba3-4ddc-ba3b-21c5d6e70c53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:14.779646 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c4d20-7f2c-41f8-8778-ee6023099b28 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:14.780977 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d073571a-fba3-4ddc-ba3b-21c5d6e70c53 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:14.790539 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 795/1583] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:14.791423 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c4d20-7f2c-41f8-8778-ee6023099b28 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:14.801213 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c4d20-7f2c-41f8-8778-ee6023099b28 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:14.806475 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f1081-f89c-45cc-9c01-b4ebf838b995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:14.807042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f1081-f89c-45cc-9c01-b4ebf838b995 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:14.807328 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f1081-f89c-45cc-9c01-b4ebf838b995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:14.807605 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f1081-f89c-45cc-9c01-b4ebf838b995 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:14.808269 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f1081-f89c-45cc-9c01-b4ebf838b995 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:14.809902 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f1081-f89c-45cc-9c01-b4ebf838b995 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:14.820340 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 796/1584] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:14.820845 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f1081-f89c-45cc-9c01-b4ebf838b995 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:14.830602 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f1081-f89c-45cc-9c01-b4ebf838b995 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:14.835486 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e9b29-27be-4cf2-832d-4f968c1bb44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:14.836621 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e9b29-27be-4cf2-832d-4f968c1bb44f admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:14.836901 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e9b29-27be-4cf2-832d-4f968c1bb44f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:14.837171 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e9b29-27be-4cf2-832d-4f968c1bb44f admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:14.837778 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3e9b29-27be-4cf2-832d-4f968c1bb44f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:14.839639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3e9b29-27be-4cf2-832d-4f968c1bb44f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:14.867794 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 789/1585] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:05:14 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:14.868815 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073571a-fba3-4ddc-ba3b-21c5d6e70c53 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:14.923335 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073571a-fba3-4ddc-ba3b-21c5d6e70c53 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:14.927310 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b8932-dc1d-42e5-b264-1470b47f09c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:14.931307 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b8932-dc1d-42e5-b264-1470b47f09c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:14.931436 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 797/1586] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:05:14 2022] POST /identity/v3/users => generated 283 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:14.931532 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b8932-dc1d-42e5-b264-1470b47f09c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:14.931532 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b8932-dc1d-42e5-b264-1470b47f09c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:14.931690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e9b29-27be-4cf2-832d-4f968c1bb44f admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:14.938969 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b8932-dc1d-42e5-b264-1470b47f09c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:14.939620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e9b29-27be-4cf2-832d-4f968c1bb44f admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:14.940768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b8932-dc1d-42e5-b264-1470b47f09c6 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:14.944247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd494ab-da21-415d-b687-a28edf32549c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:14.944837 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd494ab-da21-415d-b687-a28edf32549c admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:14.945113 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd494ab-da21-415d-b687-a28edf32549c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:14.945382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd494ab-da21-415d-b687-a28edf32549c admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:14.946016 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd494ab-da21-415d-b687-a28edf32549c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:14.947597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd494ab-da21-415d-b687-a28edf32549c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:14.950748 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 790/1587] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:14.951534 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b8932-dc1d-42e5-b264-1470b47f09c6 service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:14.958162 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 798/1588] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:14.959768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd494ab-da21-415d-b687-a28edf32549c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:14.962983 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b8932-dc1d-42e5-b264-1470b47f09c6 service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:14.969066 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1be9c-1323-4405-aa1a-1d5daa01ea00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:14.969846 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd494ab-da21-415d-b687-a28edf32549c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:14.970116 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1be9c-1323-4405-aa1a-1d5daa01ea00 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:14.970580 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1be9c-1323-4405-aa1a-1d5daa01ea00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:14.971094 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1be9c-1323-4405-aa1a-1d5daa01ea00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:14.971959 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1be9c-1323-4405-aa1a-1d5daa01ea00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:14.973741 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1be9c-1323-4405-aa1a-1d5daa01ea00 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:14.974728 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f873a81-eb5e-4d40-88fd-bfaf39ec02bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:14.975322 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f873a81-eb5e-4d40-88fd-bfaf39ec02bf admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:14.975600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f873a81-eb5e-4d40-88fd-bfaf39ec02bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:14.975870 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f873a81-eb5e-4d40-88fd-bfaf39ec02bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:14.977114 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f873a81-eb5e-4d40-88fd-bfaf39ec02bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:14.978719 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f873a81-eb5e-4d40-88fd-bfaf39ec02bf admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:14.989580 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 799/1589] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:14.995094 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f873a81-eb5e-4d40-88fd-bfaf39ec02bf admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:14.995203 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 791/1590] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:14.995203 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1be9c-1323-4405-aa1a-1d5daa01ea00 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:15.001333 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f873a81-eb5e-4d40-88fd-bfaf39ec02bf admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:15.001983 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1be9c-1323-4405-aa1a-1d5daa01ea00 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:15.005827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce3e86f-5945-486a-a151-cbc30c903a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:15.006406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3e86f-5945-486a-a151-cbc30c903a01 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:15.006701 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3e86f-5945-486a-a151-cbc30c903a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:15.006973 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3e86f-5945-486a-a151-cbc30c903a01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:15.007086 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9f458-43a5-444f-a91b-aeefe5ee60b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:15.007630 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce3e86f-5945-486a-a151-cbc30c903a01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:15.007738 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9f458-43a5-444f-a91b-aeefe5ee60b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:15.007974 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9f458-43a5-444f-a91b-aeefe5ee60b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:15.008303 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9f458-43a5-444f-a91b-aeefe5ee60b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:15.008974 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d9f458-43a5-444f-a91b-aeefe5ee60b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:15.009237 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce3e86f-5945-486a-a151-cbc30c903a01 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:15.010601 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d9f458-43a5-444f-a91b-aeefe5ee60b1 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:15.010729 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:15.019917 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 800/1591] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:15.024482 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce3e86f-5945-486a-a151-cbc30c903a01 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:15.024782 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 792/1592] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:15.024782 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9f458-43a5-444f-a91b-aeefe5ee60b1 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:15.031123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce3e86f-5945-486a-a151-cbc30c903a01 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:15.034280 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9f458-43a5-444f-a91b-aeefe5ee60b1 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:15.035697 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24a685-1a25-4f0d-a9a5-5985207d0c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:15.036323 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24a685-1a25-4f0d-a9a5-5985207d0c98 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:15.036599 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24a685-1a25-4f0d-a9a5-5985207d0c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:15.036871 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24a685-1a25-4f0d-a9a5-5985207d0c98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:15.037498 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d24a685-1a25-4f0d-a9a5-5985207d0c98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:15.038862 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b793bca-672a-461a-87f0-f8d514b7fe50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:15.039707 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d24a685-1a25-4f0d-a9a5-5985207d0c98 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:15.040171 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b793bca-672a-461a-87f0-f8d514b7fe50 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:15.044979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b793bca-672a-461a-87f0-f8d514b7fe50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:15.044979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b793bca-672a-461a-87f0-f8d514b7fe50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:15.044979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b793bca-672a-461a-87f0-f8d514b7fe50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:15.044979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b793bca-672a-461a-87f0-f8d514b7fe50 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:15.050176 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 801/1593] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:15.051308 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24a685-1a25-4f0d-a9a5-5985207d0c98 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:15.059646 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 793/1594] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:15.059646 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b793bca-672a-461a-87f0-f8d514b7fe50 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:15.065660 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24a685-1a25-4f0d-a9a5-5985207d0c98 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:15.067650 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b793bca-672a-461a-87f0-f8d514b7fe50 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:15.071160 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615fb1ae-be45-42aa-952a-cbf772384e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:15.071700 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615fb1ae-be45-42aa-952a-cbf772384e20 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:15.071978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615fb1ae-be45-42aa-952a-cbf772384e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:15.072524 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615fb1ae-be45-42aa-952a-cbf772384e20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:15.072796 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf07c0e-d5ca-4199-a7af-cc1ce67b959b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:15.073196 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615fb1ae-be45-42aa-952a-cbf772384e20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:15.073504 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf07c0e-d5ca-4199-a7af-cc1ce67b959b admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:15.073931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf07c0e-d5ca-4199-a7af-cc1ce67b959b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:15.074204 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf07c0e-d5ca-4199-a7af-cc1ce67b959b admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:15.074801 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615fb1ae-be45-42aa-952a-cbf772384e20 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:15.075176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf07c0e-d5ca-4199-a7af-cc1ce67b959b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:15.076716 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf07c0e-d5ca-4199-a7af-cc1ce67b959b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:15.090137 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 802/1595] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:15.090137 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615fb1ae-be45-42aa-952a-cbf772384e20 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:15.090442 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 794/1596] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:15.090442 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf07c0e-d5ca-4199-a7af-cc1ce67b959b admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:15.100820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615fb1ae-be45-42aa-952a-cbf772384e20 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:15.103447 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf07c0e-d5ca-4199-a7af-cc1ce67b959b admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:15.106322 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff379d2-edb3-4ba3-b7a4-e44e5732097d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:15.108124 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff379d2-edb3-4ba3-b7a4-e44e5732097d admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:15.108124 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff379d2-edb3-4ba3-b7a4-e44e5732097d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:15.108124 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff379d2-edb3-4ba3-b7a4-e44e5732097d admin admin] PATH_INFO: `/v3/projects/565a50447ba547cb871ff2365b635131/users/4377c9e00e3e4ccbb81a988d74291d70/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:15.112423 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f5f23e-5a8f-4bc8-9210-f92dfc681f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:15.113747 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5f23e-5a8f-4bc8-9210-f92dfc681f8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:15.114124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5f23e-5a8f-4bc8-9210-f92dfc681f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:15.114390 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5f23e-5a8f-4bc8-9210-f92dfc681f8d admin admin] PATH_INFO: `/v3/projects/620d0a2c4566408aa134f2a8ac7d91ee/users/710e49db188845bbae5a183c6657e07f/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:15.124518 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff379d2-edb3-4ba3-b7a4-e44e5732097d admin admin] RBAC: Authorizing `identity:create_grant(project_id=565a50447ba547cb871ff2365b635131, user_id=4377c9e00e3e4ccbb81a988d74291d70, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:15.127108 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff379d2-edb3-4ba3-b7a4-e44e5732097d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:15.130330 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f5f23e-5a8f-4bc8-9210-f92dfc681f8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=620d0a2c4566408aa134f2a8ac7d91ee, user_id=710e49db188845bbae5a183c6657e07f, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:15.133599 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f5f23e-5a8f-4bc8-9210-f92dfc681f8d admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:15.214600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 803/1597] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:15 2022] PUT /identity/v3/projects/565a50447ba547cb871ff2365b635131/users/4377c9e00e3e4ccbb81a988d74291d70/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:15.215375 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 795/1598] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:15 2022] PUT /identity/v3/projects/620d0a2c4566408aa134f2a8ac7d91ee/users/710e49db188845bbae5a183c6657e07f/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:15.215493 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff379d2-edb3-4ba3-b7a4-e44e5732097d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:15.216603 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f5f23e-5a8f-4bc8-9210-f92dfc681f8d admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:15.248139 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:15.290465 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff379d2-edb3-4ba3-b7a4-e44e5732097d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:15.292883 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f5f23e-5a8f-4bc8-9210-f92dfc681f8d admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:15.295932 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f71dd-37d3-4526-9d9d-ea0dafda4d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:15.296806 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f71dd-37d3-4526-9d9d-ea0dafda4d75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:15.302398 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f71dd-37d3-4526-9d9d-ea0dafda4d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:15.302398 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f71dd-37d3-4526-9d9d-ea0dafda4d75 admin admin] PATH_INFO: `/v3/projects/d9f70ceb84fc40fbb499b6a80bc5c13b/users/e3ef96f764244e5893f16a74d4169edb/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:15.302589 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7875752e-e993-493b-8c1d-5ce024df7746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:15.302589 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875752e-e993-493b-8c1d-5ce024df7746 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:15.302589 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875752e-e993-493b-8c1d-5ce024df7746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:15.302589 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875752e-e993-493b-8c1d-5ce024df7746 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:15.302893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7875752e-e993-493b-8c1d-5ce024df7746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:15.302893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7875752e-e993-493b-8c1d-5ce024df7746 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:15.313958 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 796/1599] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:15.314809 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7875752e-e993-493b-8c1d-5ce024df7746 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:15.317413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f71dd-37d3-4526-9d9d-ea0dafda4d75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9f70ceb84fc40fbb499b6a80bc5c13b, user_id=e3ef96f764244e5893f16a74d4169edb, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:15.320202 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f71dd-37d3-4526-9d9d-ea0dafda4d75 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:15.330926 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7875752e-e993-493b-8c1d-5ce024df7746 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:15.331169 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a1967-e06a-4f57-816d-d85a33000392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:15.331759 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a1967-e06a-4f57-816d-d85a33000392 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:15.332039 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a1967-e06a-4f57-816d-d85a33000392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:15.332393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a1967-e06a-4f57-816d-d85a33000392 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:15.332986 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03a1967-e06a-4f57-816d-d85a33000392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:15.334571 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03a1967-e06a-4f57-816d-d85a33000392 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:15.344778 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 797/1600] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:15.345387 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a1967-e06a-4f57-816d-d85a33000392 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:15.355574 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a1967-e06a-4f57-816d-d85a33000392 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:15.360317 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc90073-2f1f-4cf2-b041-6e233adb0c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:15.360932 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc90073-2f1f-4cf2-b041-6e233adb0c9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:15.361209 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc90073-2f1f-4cf2-b041-6e233adb0c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:15.361481 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc90073-2f1f-4cf2-b041-6e233adb0c9b admin admin] PATH_INFO: `/v3/projects/4e7003778bfe41bbb0571fa23db64b7a/users/066eeca5e1e64725ab8081c34121b3ce/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:15.376187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc90073-2f1f-4cf2-b041-6e233adb0c9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e7003778bfe41bbb0571fa23db64b7a, user_id=066eeca5e1e64725ab8081c34121b3ce, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:15.378857 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc90073-2f1f-4cf2-b041-6e233adb0c9b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:15.412867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 804/1601] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:15 2022] PUT /identity/v3/projects/d9f70ceb84fc40fbb499b6a80bc5c13b/users/e3ef96f764244e5893f16a74d4169edb/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:15.414888 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f71dd-37d3-4526-9d9d-ea0dafda4d75 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:15.466407 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 798/1602] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:15 2022] PUT /identity/v3/projects/4e7003778bfe41bbb0571fa23db64b7a/users/066eeca5e1e64725ab8081c34121b3ce/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:15.468116 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad693b-951a-43bb-a464-e4df7bba6413 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:15.468544 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad693b-951a-43bb-a464-e4df7bba6413 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:15.468851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad693b-951a-43bb-a464-e4df7bba6413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:15.490410 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f71dd-37d3-4526-9d9d-ea0dafda4d75 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:15.494549 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-4bad693b-951a-43bb-a464-e4df7bba6413 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:15.494886 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-4bad693b-951a-43bb-a464-e4df7bba6413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:15.530939 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:15.554233 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d034b3-ed05-4370-9101-da3c0de94627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:15.555479 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d034b3-ed05-4370-9101-da3c0de94627 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:15.556554 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d034b3-ed05-4370-9101-da3c0de94627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:15.556673 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d034b3-ed05-4370-9101-da3c0de94627 admin admin] PATH_INFO: `/v3/projects/46978366f11c4af79a9e9ad6ff2e5e05/users/83e9308215e94c5e9e615be1f2a113b1/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:15.577292 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d034b3-ed05-4370-9101-da3c0de94627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46978366f11c4af79a9e9ad6ff2e5e05, user_id=83e9308215e94c5e9e615be1f2a113b1, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:15.577292 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d034b3-ed05-4370-9101-da3c0de94627 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:15.593428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 799/1603] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:15.593428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce43c32-067a-4af4-96bb-16cd6931e861 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:15.593428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce43c32-067a-4af4-96bb-16cd6931e861 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:15.593428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce43c32-067a-4af4-96bb-16cd6931e861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:15.621309 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-fce43c32-067a-4af4-96bb-16cd6931e861 None None] MFA Rules not processed for user `4377c9e00e3e4ccbb81a988d74291d70`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:15.622509 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-fce43c32-067a-4af4-96bb-16cd6931e861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:15.665107 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 805/1604] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:15 2022] PUT /identity/v3/projects/46978366f11c4af79a9e9ad6ff2e5e05/users/83e9308215e94c5e9e615be1f2a113b1/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:15.666109 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d034b3-ed05-4370-9101-da3c0de94627 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:15.691110 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 800/1605] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:15.691925 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce43c32-067a-4af4-96bb-16cd6931e861 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:15.703499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce43c32-067a-4af4-96bb-16cd6931e861 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:15.716824 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:15.737042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d034b3-ed05-4370-9101-da3c0de94627 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:15.742317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700ea19-4fc7-4144-8676-cdd806bf26ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:15.743208 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700ea19-4fc7-4144-8676-cdd806bf26ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:15.743572 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700ea19-4fc7-4144-8676-cdd806bf26ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:15.743911 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700ea19-4fc7-4144-8676-cdd806bf26ad admin admin] PATH_INFO: `/v3/projects/f1eaf427c6044461a8000994e2f4832c/users/2f9fd461fa8840dbb1123e14db7740f3/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:15.764219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7700ea19-4fc7-4144-8676-cdd806bf26ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1eaf427c6044461a8000994e2f4832c, user_id=2f9fd461fa8840dbb1123e14db7740f3, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:15.767582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7700ea19-4fc7-4144-8676-cdd806bf26ad admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:15.774280 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ec7ce-b6cb-4dea-92f5-7c06d2631d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:15.774960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ec7ce-b6cb-4dea-92f5-7c06d2631d56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:15.775267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ec7ce-b6cb-4dea-92f5-7c06d2631d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:15.775540 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ec7ce-b6cb-4dea-92f5-7c06d2631d56 admin admin] PATH_INFO: `/v3/projects/c061b12a075e4876a2a455cc048442ad/users/2faaa45412584cb2b4adf0e37d77eafd/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:15.791385 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ec7ce-b6cb-4dea-92f5-7c06d2631d56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c061b12a075e4876a2a455cc048442ad, user_id=2faaa45412584cb2b4adf0e37d77eafd, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:15.794443 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ec7ce-b6cb-4dea-92f5-7c06d2631d56 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:15.915113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 801/1606] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:15 2022] PUT /identity/v3/projects/c061b12a075e4876a2a455cc048442ad/users/2faaa45412584cb2b4adf0e37d77eafd/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:15.915790 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 806/1607] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:15 2022] PUT /identity/v3/projects/f1eaf427c6044461a8000994e2f4832c/users/2f9fd461fa8840dbb1123e14db7740f3/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:15.916257 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907d0a2-d86b-4d6f-af97-963d54c779ec None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:15.916503 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745eb670-724d-4fad-a705-003e607d5c7a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:15.916621 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907d0a2-d86b-4d6f-af97-963d54c779ec None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:15.916856 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745eb670-724d-4fad-a705-003e607d5c7a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:15.916966 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907d0a2-d86b-4d6f-af97-963d54c779ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:15.917196 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745eb670-724d-4fad-a705-003e607d5c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:15.944226 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-745eb670-724d-4fad-a705-003e607d5c7a None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:15.946176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-745eb670-724d-4fad-a705-003e607d5c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:15.946967 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-3907d0a2-d86b-4d6f-af97-963d54c779ec None None] MFA Rules not processed for user `066eeca5e1e64725ab8081c34121b3ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:15.948184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3907d0a2-d86b-4d6f-af97-963d54c779ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:16.014790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 802/1608] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:16.015487 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745eb670-724d-4fad-a705-003e607d5c7a None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:16.032105 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 807/1609] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:16.047894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05addf1e-f1ac-45e5-b4db-3f20d5307834 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:16.047894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05addf1e-f1ac-45e5-b4db-3f20d5307834 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:16.047894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05addf1e-f1ac-45e5-b4db-3f20d5307834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:16.068255 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-05addf1e-f1ac-45e5-b4db-3f20d5307834 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:16.070662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-05addf1e-f1ac-45e5-b4db-3f20d5307834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:16.083310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 808/1610] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:16.084422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05addf1e-f1ac-45e5-b4db-3f20d5307834 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:16.095495 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05addf1e-f1ac-45e5-b4db-3f20d5307834 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:16.097562 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745eb670-724d-4fad-a705-003e607d5c7a None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:16.101520 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b238c6-5626-44c6-8b3a-751e3fee35a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:16.102131 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b238c6-5626-44c6-8b3a-751e3fee35a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:16.102438 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b238c6-5626-44c6-8b3a-751e3fee35a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:16.102765 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b238c6-5626-44c6-8b3a-751e3fee35a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:16.107995 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852eeddf-ddf9-4908-8d99-25e8c04479bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:16.107995 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852eeddf-ddf9-4908-8d99-25e8c04479bd admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:16.107995 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852eeddf-ddf9-4908-8d99-25e8c04479bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:16.107995 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852eeddf-ddf9-4908-8d99-25e8c04479bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:16.108403 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852eeddf-ddf9-4908-8d99-25e8c04479bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:16.108403 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852eeddf-ddf9-4908-8d99-25e8c04479bd admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:16.112317 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b238c6-5626-44c6-8b3a-751e3fee35a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:16.113863 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b238c6-5626-44c6-8b3a-751e3fee35a5 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:16.123360 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 803/1611] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:16.124686 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59cd7e9-d3bf-4d1e-be0e-34635db5741f None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:16.124979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59cd7e9-d3bf-4d1e-be0e-34635db5741f None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:16.125285 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59cd7e9-d3bf-4d1e-be0e-34635db5741f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:16.154297 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-d59cd7e9-d3bf-4d1e-be0e-34635db5741f None None] MFA Rules not processed for user `2faaa45412584cb2b4adf0e37d77eafd`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:16.155487 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d59cd7e9-d3bf-4d1e-be0e-34635db5741f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:16.184336 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:16.218668 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 804/1612] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:16.220055 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94ffef-55e7-42fc-b35e-7d1752ae78cc None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:16.220371 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94ffef-55e7-42fc-b35e-7d1752ae78cc None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:16.220645 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94ffef-55e7-42fc-b35e-7d1752ae78cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:16.251285 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 809/1613] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:05:16 2022] POST /identity/v3/projects => generated 371 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:16.251285 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852eeddf-ddf9-4908-8d99-25e8c04479bd admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:16.251673 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-5d94ffef-55e7-42fc-b35e-7d1752ae78cc None None] MFA Rules not processed for user `2f9fd461fa8840dbb1123e14db7740f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:16.251673 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5d94ffef-55e7-42fc-b35e-7d1752ae78cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:16.262099 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:16.340897 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 805/1614] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:16.341723 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3479e-f2d2-48be-a553-6d25e3961dba None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:16.342013 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3479e-f2d2-48be-a553-6d25e3961dba None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:16.342418 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3479e-f2d2-48be-a553-6d25e3961dba None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:16.343161 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852eeddf-ddf9-4908-8d99-25e8c04479bd admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:16.343920 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 806/1615] 10.222.0.48 () {54 vars in 962 bytes} [Fri Sep 9 21:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:05:16.349737 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b56ae-aab4-405f-b08c-1d2f49eeb457 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:16.350908 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b56ae-aab4-405f-b08c-1d2f49eeb457 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:16.352020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b56ae-aab4-405f-b08c-1d2f49eeb457 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:16.352616 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b56ae-aab4-405f-b08c-1d2f49eeb457 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:16.362201 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d3479e-f2d2-48be-a553-6d25e3961dba None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:16.388446 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d3479e-f2d2-48be-a553-6d25e3961dba None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:16.401318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65a002-c8e8-4605-b13d-0bbafbefae4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:16.401997 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65a002-c8e8-4605-b13d-0bbafbefae4c service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:16.402172 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65a002-c8e8-4605-b13d-0bbafbefae4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:16.402444 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65a002-c8e8-4605-b13d-0bbafbefae4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:16.446851 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16b56ae-aab4-405f-b08c-1d2f49eeb457 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:16.448539 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16b56ae-aab4-405f-b08c-1d2f49eeb457 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:16.460267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 810/1616] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:16.461222 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b56ae-aab4-405f-b08c-1d2f49eeb457 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:16.495366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65a002-c8e8-4605-b13d-0bbafbefae4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:16.495366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65a002-c8e8-4605-b13d-0bbafbefae4c service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:16.502259 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 807/1617] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:16.503120 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65a002-c8e8-4605-b13d-0bbafbefae4c service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:16.515972 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65a002-c8e8-4605-b13d-0bbafbefae4c service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:16.523446 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f6fb7-823f-4b2b-903a-7a57d48b94af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:16.523884 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f6fb7-823f-4b2b-903a-7a57d48b94af admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:16.524308 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f6fb7-823f-4b2b-903a-7a57d48b94af admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:16.524648 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f6fb7-823f-4b2b-903a-7a57d48b94af admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:16.525356 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f6fb7-823f-4b2b-903a-7a57d48b94af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:16.527660 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f6fb7-823f-4b2b-903a-7a57d48b94af admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:16.536756 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b56ae-aab4-405f-b08c-1d2f49eeb457 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:16.543310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f463157-2ad4-45cf-a42d-c401304faf6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:16.545474 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f463157-2ad4-45cf-a42d-c401304faf6c service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:16.545474 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f463157-2ad4-45cf-a42d-c401304faf6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:16.545474 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f463157-2ad4-45cf-a42d-c401304faf6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:16.612721 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f463157-2ad4-45cf-a42d-c401304faf6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:16.614173 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f463157-2ad4-45cf-a42d-c401304faf6c service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:16.626567 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 811/1618] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:16.631614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757877e-af65-4ad8-b64a-6fa6106868d4 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:16.631614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757877e-af65-4ad8-b64a-6fa6106868d4 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:16.631614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757877e-af65-4ad8-b64a-6fa6106868d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:16.652479 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 808/1619] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:05:16 2022] POST /identity/v3/users => generated 292 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:16.660404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67978ff6-b2a3-40c2-8654-4efd7692177d None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:16.660404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67978ff6-b2a3-40c2-8654-4efd7692177d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:16.660404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67978ff6-b2a3-40c2-8654-4efd7692177d None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:16.660707 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 809/1620] 10.222.0.48 () {54 vars in 962 bytes} [Fri Sep 9 21:05:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:05:16.669315 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67978ff6-b2a3-40c2-8654-4efd7692177d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:16.678656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-0757877e-af65-4ad8-b64a-6fa6106868d4 None None] MFA Rules not processed for user `613b845a9b4d432daf8d478fc3970708`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:16.684014 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0757877e-af65-4ad8-b64a-6fa6106868d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:16.691951 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67978ff6-b2a3-40c2-8654-4efd7692177d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:16.697175 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155c41f-577d-40b9-9eb4-d91f9af704a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:16.697774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155c41f-577d-40b9-9eb4-d91f9af704a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:16.698047 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155c41f-577d-40b9-9eb4-d91f9af704a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:16.698315 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155c41f-577d-40b9-9eb4-d91f9af704a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:16.698959 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3155c41f-577d-40b9-9eb4-d91f9af704a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:16.700847 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3155c41f-577d-40b9-9eb4-d91f9af704a3 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:16.715059 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 810/1621] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:16.717795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0529cc2-240b-4fc0-a2ef-d3685d26eb9b None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:16.717795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0529cc2-240b-4fc0-a2ef-d3685d26eb9b None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:16.717795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0529cc2-240b-4fc0-a2ef-d3685d26eb9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:16.739544 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-c0529cc2-240b-4fc0-a2ef-d3685d26eb9b None None] MFA Rules not processed for user `613b845a9b4d432daf8d478fc3970708`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:16.741987 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c0529cc2-240b-4fc0-a2ef-d3685d26eb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:16.797225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 812/1622] 10.222.0.48 () {58 vars in 1059 bytes} [Fri Sep 9 21:05:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:16.798127 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757877e-af65-4ad8-b64a-6fa6106868d4 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:16.812793 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757877e-af65-4ad8-b64a-6fa6106868d4 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:16.825318 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208e89e-7b00-4d7d-8481-77c78dbf81ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:16.826335 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208e89e-7b00-4d7d-8481-77c78dbf81ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:16.826834 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208e89e-7b00-4d7d-8481-77c78dbf81ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:16.827597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208e89e-7b00-4d7d-8481-77c78dbf81ec admin admin] PATH_INFO: `/v3/projects/82df673d62fb411e9b0f9725f173c3f6/users/27d52d2e932b42ca97ea52289186d1a1/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:16.854108 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6208e89e-7b00-4d7d-8481-77c78dbf81ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=82df673d62fb411e9b0f9725f173c3f6, user_id=27d52d2e932b42ca97ea52289186d1a1, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:16.857219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6208e89e-7b00-4d7d-8481-77c78dbf81ec admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:16.860959 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 811/1623] 10.222.0.48 () {58 vars in 1059 bytes} [Fri Sep 9 21:05:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:16.861860 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0529cc2-240b-4fc0-a2ef-d3685d26eb9b None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:16.875662 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0529cc2-240b-4fc0-a2ef-d3685d26eb9b None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:16.887175 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22342a62-45f9-4e63-9784-410b3f59ddba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:16.887797 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22342a62-45f9-4e63-9784-410b3f59ddba service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:16.888096 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22342a62-45f9-4e63-9784-410b3f59ddba service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:16.888441 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22342a62-45f9-4e63-9784-410b3f59ddba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:16.963273 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22342a62-45f9-4e63-9784-410b3f59ddba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:16.965647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22342a62-45f9-4e63-9784-410b3f59ddba service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:16.969997 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 813/1624] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:16 2022] PUT /identity/v3/projects/82df673d62fb411e9b0f9725f173c3f6/users/27d52d2e932b42ca97ea52289186d1a1/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:16.971044 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208e89e-7b00-4d7d-8481-77c78dbf81ec admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:16.978138 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 812/1625] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:16.979056 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22342a62-45f9-4e63-9784-410b3f59ddba service nova] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:16.989411 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:16.995907 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22342a62-45f9-4e63-9784-410b3f59ddba service nova] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:17.082744 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b0c5d-580e-44cd-bf74-5bc1fb24a887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:17.083884 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b0c5d-580e-44cd-bf74-5bc1fb24a887 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:17.084791 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b0c5d-580e-44cd-bf74-5bc1fb24a887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:17.085084 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b0c5d-580e-44cd-bf74-5bc1fb24a887 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:17.086714 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843b0c5d-580e-44cd-bf74-5bc1fb24a887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:17.089262 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843b0c5d-580e-44cd-bf74-5bc1fb24a887 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:17.106184 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 813/1626] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:17.110333 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b0c5d-580e-44cd-bf74-5bc1fb24a887 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:17.126046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b0c5d-580e-44cd-bf74-5bc1fb24a887 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:17.133144 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05444296-1fde-4499-9cbe-84bea2794ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:17.134263 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05444296-1fde-4499-9cbe-84bea2794ec5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:17.134723 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05444296-1fde-4499-9cbe-84bea2794ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:17.135280 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05444296-1fde-4499-9cbe-84bea2794ec5 admin admin] PATH_INFO: `/v3/projects/82df673d62fb411e9b0f9725f173c3f6/users/27d52d2e932b42ca97ea52289186d1a1/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:17.139705 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05444296-1fde-4499-9cbe-84bea2794ec5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82df673d62fb411e9b0f9725f173c3f6, user_id=27d52d2e932b42ca97ea52289186d1a1, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:17.142308 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208e89e-7b00-4d7d-8481-77c78dbf81ec admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:17.149724 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05444296-1fde-4499-9cbe-84bea2794ec5 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:17.158717 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 814/1627] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:17 2022] PUT /identity/v3/projects/82df673d62fb411e9b0f9725f173c3f6/users/27d52d2e932b42ca97ea52289186d1a1/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:17.169586 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce93e1b-1af9-48fc-9893-83f40efee8ec None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:17.170186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce93e1b-1af9-48fc-9893-83f40efee8ec None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:17.170498 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce93e1b-1af9-48fc-9893-83f40efee8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:17.172967 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a1f56-a91f-45ef-a430-f857cacb8f47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:17.173468 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a1f56-a91f-45ef-a430-f857cacb8f47 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:17.173751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a1f56-a91f-45ef-a430-f857cacb8f47 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:17.174028 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a1f56-a91f-45ef-a430-f857cacb8f47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:17.252517 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-9ce93e1b-1af9-48fc-9893-83f40efee8ec None None] MFA Rules not processed for user `27d52d2e932b42ca97ea52289186d1a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:17.256023 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-9ce93e1b-1af9-48fc-9893-83f40efee8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:17.266219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:17.291028 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a1f56-a91f-45ef-a430-f857cacb8f47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:17.292702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a1f56-a91f-45ef-a430-f857cacb8f47 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:17.311390 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 814/1628] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:17.386916 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 815/1629] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:17 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:17.406394 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a1f56-a91f-45ef-a430-f857cacb8f47 service nova] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:17.468754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd60f02-4f34-48af-b720-a446f9d1c3a3 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:17.468978 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd60f02-4f34-48af-b720-a446f9d1c3a3 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:17.469161 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd60f02-4f34-48af-b720-a446f9d1c3a3 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:17.470678 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 816/1630] 10.222.0.48 () {54 vars in 912 bytes} [Fri Sep 9 21:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:05:17.474451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:17.477270 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1681a3-60a4-447f-9b28-a7fd4bce0877 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:17.477270 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1681a3-60a4-447f-9b28-a7fd4bce0877 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:17.477270 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1681a3-60a4-447f-9b28-a7fd4bce0877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:17.519287 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-3c1681a3-60a4-447f-9b28-a7fd4bce0877 None None] MFA Rules not processed for user `613b845a9b4d432daf8d478fc3970708`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:17.523123 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3c1681a3-60a4-447f-9b28-a7fd4bce0877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:17.530115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a1f56-a91f-45ef-a430-f857cacb8f47 service nova] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:17.672701 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02efff83-bbba-453c-aa23-927bc4246581 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:17.672701 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02efff83-bbba-453c-aa23-927bc4246581 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:17.675700 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02efff83-bbba-453c-aa23-927bc4246581 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:17.675700 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02efff83-bbba-453c-aa23-927bc4246581 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:17.694132 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:17.704304 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02efff83-bbba-453c-aa23-927bc4246581 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:17.707055 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02efff83-bbba-453c-aa23-927bc4246581 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:17.714923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 817/1631] 10.222.0.48 () {58 vars in 1009 bytes} [Fri Sep 9 21:05:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:05:17.719714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 815/1632] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:17 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:17.733278 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637d5ef-d5e6-48ad-85ad-71ca2a156742 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:17.733278 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637d5ef-d5e6-48ad-85ad-71ca2a156742 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:17.733278 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637d5ef-d5e6-48ad-85ad-71ca2a156742 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:17.736632 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 818/1633] 10.222.0.48 () {54 vars in 912 bytes} [Fri Sep 9 21:05:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:05:17.743060 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c1288-e295-4e2d-8f6b-49006590b9d7 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:17.743680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c1288-e295-4e2d-8f6b-49006590b9d7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:17.745204 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c1288-e295-4e2d-8f6b-49006590b9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:17.897815 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-046c1288-e295-4e2d-8f6b-49006590b9d7 None None] MFA Rules not processed for user `613b845a9b4d432daf8d478fc3970708`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:17.897815 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-046c1288-e295-4e2d-8f6b-49006590b9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:17.897815 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 816/1634] 10.222.0.48 () {58 vars in 1009 bytes} [Fri Sep 9 21:05:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:17.900326 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637d5ef-d5e6-48ad-85ad-71ca2a156742 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:17.925675 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637d5ef-d5e6-48ad-85ad-71ca2a156742 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:17.927684 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05368acd-1b24-4230-a251-2908e618e20b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:17.928665 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05368acd-1b24-4230-a251-2908e618e20b service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:17.929265 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05368acd-1b24-4230-a251-2908e618e20b service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:17.929831 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05368acd-1b24-4230-a251-2908e618e20b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:17.940224 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c1288-e295-4e2d-8f6b-49006590b9d7 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:17.952348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c1288-e295-4e2d-8f6b-49006590b9d7 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:17.956622 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa8c27-7dfc-479b-a2d5-6313e19eb270 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:17.959889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa8c27-7dfc-479b-a2d5-6313e19eb270 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:17.959889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa8c27-7dfc-479b-a2d5-6313e19eb270 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:17.959889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa8c27-7dfc-479b-a2d5-6313e19eb270 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:17.964738 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05368acd-1b24-4230-a251-2908e618e20b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:17.968670 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05368acd-1b24-4230-a251-2908e618e20b service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:17.976154 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa8c27-7dfc-479b-a2d5-6313e19eb270 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:17.978883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa8c27-7dfc-479b-a2d5-6313e19eb270 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:17.992274 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 817/1635] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:18.000989 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 819/1636] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:18.786931 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa8c27-7dfc-479b-a2d5-6313e19eb270 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:18.876747 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa8c27-7dfc-479b-a2d5-6313e19eb270 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:18.886804 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed6cfba-9bbd-4790-abbf-8f74bfd2f4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:18.887582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6cfba-9bbd-4790-abbf-8f74bfd2f4a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:18.887938 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6cfba-9bbd-4790-abbf-8f74bfd2f4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:18.888356 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6cfba-9bbd-4790-abbf-8f74bfd2f4a6 admin admin] PATH_INFO: `/v3/users/2f9fd461fa8840dbb1123e14db7740f3` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:18.890649 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed6cfba-9bbd-4790-abbf-8f74bfd2f4a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f9fd461fa8840dbb1123e14db7740f3)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:18.894558 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed6cfba-9bbd-4790-abbf-8f74bfd2f4a6 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:19.243751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:19.247767 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-eed6cfba-9bbd-4790-abbf-8f74bfd2f4a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f9fd461fa8840dbb1123e14db7740f3'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:19.248670 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-eed6cfba-9bbd-4790-abbf-8f74bfd2f4a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f9fd461fa8840dbb1123e14db7740f3'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:19.249591 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-eed6cfba-9bbd-4790-abbf-8f74bfd2f4a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f9fd461fa8840dbb1123e14db7740f3'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:19.257963 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-eed6cfba-9bbd-4790-abbf-8f74bfd2f4a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f9fd461fa8840dbb1123e14db7740f3'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:19.274383 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-eed6cfba-9bbd-4790-abbf-8f74bfd2f4a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f9fd461fa8840dbb1123e14db7740f3'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:19.417472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 818/1637] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:05:18 2022] DELETE /identity/v3/users/2f9fd461fa8840dbb1123e14db7740f3 => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:19.421743 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f7f98-f668-49b9-84ef-efd8e89f05c1 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:19.422568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f7f98-f668-49b9-84ef-efd8e89f05c1 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:19.422998 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f7f98-f668-49b9-84ef-efd8e89f05c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:19.487208 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-ae8f7f98-f668-49b9-84ef-efd8e89f05c1 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:19.488509 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ae8f7f98-f668-49b9-84ef-efd8e89f05c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:19.616425 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 820/1638] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:19.717597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed6cfba-9bbd-4790-abbf-8f74bfd2f4a6 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:19.719544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-eed6cfba-9bbd-4790-abbf-8f74bfd2f4a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:19.813954 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f7f98-f668-49b9-84ef-efd8e89f05c1 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:19.815486 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ae8f7f98-f668-49b9-84ef-efd8e89f05c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:19.840355 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f7f98-f668-49b9-84ef-efd8e89f05c1 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:19.842836 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed6cfba-9bbd-4790-abbf-8f74bfd2f4a6 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:19.853325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d63e8-c637-41da-81a5-7d119e60a3e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:19.854805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d63e8-c637-41da-81a5-7d119e60a3e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:19.855567 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d63e8-c637-41da-81a5-7d119e60a3e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:19.855837 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d63e8-c637-41da-81a5-7d119e60a3e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:19.858999 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9707cd5-0989-4aba-8dd7-d2cc262ce58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:19.860164 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9707cd5-0989-4aba-8dd7-d2cc262ce58c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:19.861314 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9707cd5-0989-4aba-8dd7-d2cc262ce58c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:19.862073 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9707cd5-0989-4aba-8dd7-d2cc262ce58c admin admin] PATH_INFO: `/v3/users/4377c9e00e3e4ccbb81a988d74291d70` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:19.864538 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9707cd5-0989-4aba-8dd7-d2cc262ce58c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4377c9e00e3e4ccbb81a988d74291d70)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:19.872543 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: WARNING py.warnings [None req-b9707cd5-0989-4aba-8dd7-d2cc262ce58c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 21:05:19.872543 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: warnings.warn(msg) Sep 09 21:05:19.872543 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: Sep 09 21:05:19.872543 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9707cd5-0989-4aba-8dd7-d2cc262ce58c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:19.878870 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d63e8-c637-41da-81a5-7d119e60a3e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:19.882214 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d63e8-c637-41da-81a5-7d119e60a3e9 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:19.913211 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 819/1639] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:20.188029 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:20.201097 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b9707cd5-0989-4aba-8dd7-d2cc262ce58c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4377c9e00e3e4ccbb81a988d74291d70'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:20.208217 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b9707cd5-0989-4aba-8dd7-d2cc262ce58c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4377c9e00e3e4ccbb81a988d74291d70'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:20.235986 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b9707cd5-0989-4aba-8dd7-d2cc262ce58c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4377c9e00e3e4ccbb81a988d74291d70'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:20.264040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b9707cd5-0989-4aba-8dd7-d2cc262ce58c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4377c9e00e3e4ccbb81a988d74291d70'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:20.265614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b9707cd5-0989-4aba-8dd7-d2cc262ce58c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4377c9e00e3e4ccbb81a988d74291d70'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:20.411818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 821/1640] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:05:19 2022] DELETE /identity/v3/users/4377c9e00e3e4ccbb81a988d74291d70 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:20.416633 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731730b-1e33-4598-bb67-a010540de57d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:20.417160 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731730b-1e33-4598-bb67-a010540de57d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:20.417160 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731730b-1e33-4598-bb67-a010540de57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:20.455792 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-2731730b-1e33-4598-bb67-a010540de57d None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:20.457098 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2731730b-1e33-4598-bb67-a010540de57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:20.696208 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 820/1641] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:20.707172 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9707cd5-0989-4aba-8dd7-d2cc262ce58c admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:20.709682 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b9707cd5-0989-4aba-8dd7-d2cc262ce58c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:20.893772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9707cd5-0989-4aba-8dd7-d2cc262ce58c admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:20.906516 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa5c63-163f-483d-be8a-380ef88b052e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:20.907369 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa5c63-163f-483d-be8a-380ef88b052e service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:20.907657 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa5c63-163f-483d-be8a-380ef88b052e service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:20.907934 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa5c63-163f-483d-be8a-380ef88b052e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:20.931066 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa5c63-163f-483d-be8a-380ef88b052e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:20.934831 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa5c63-163f-483d-be8a-380ef88b052e service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:20.967509 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 822/1642] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:22.082090 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731730b-1e33-4598-bb67-a010540de57d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:22.082090 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2731730b-1e33-4598-bb67-a010540de57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:22.098219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731730b-1e33-4598-bb67-a010540de57d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:22.112529 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de106c-a4b4-429a-8957-cd0974ddfee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:22.121624 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de106c-a4b4-429a-8957-cd0974ddfee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:22.121624 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de106c-a4b4-429a-8957-cd0974ddfee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:22.121624 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de106c-a4b4-429a-8957-cd0974ddfee7 admin admin] PATH_INFO: `/v3/projects/f1eaf427c6044461a8000994e2f4832c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:22.121624 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68de106c-a4b4-429a-8957-cd0974ddfee7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1eaf427c6044461a8000994e2f4832c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:22.121624 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: WARNING py.warnings [None req-68de106c-a4b4-429a-8957-cd0974ddfee7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 21:05:22.121624 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: warnings.warn(msg) Sep 09 21:05:22.121624 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: Sep 09 21:05:22.124247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68de106c-a4b4-429a-8957-cd0974ddfee7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:22.323461 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-68de106c-a4b4-429a-8957-cd0974ddfee7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1eaf427c6044461a8000994e2f4832c', 'request_id': 'req-68de106c-a4b4-429a-8957-cd0974ddfee7'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:22.366269 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-68de106c-a4b4-429a-8957-cd0974ddfee7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1eaf427c6044461a8000994e2f4832c', 'request_id': 'req-68de106c-a4b4-429a-8957-cd0974ddfee7'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:22.378579 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:22.386150 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa5c63-163f-483d-be8a-380ef88b052e service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:22.391466 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ebfa5c63-163f-483d-be8a-380ef88b052e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:22.416021 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa5c63-163f-483d-be8a-380ef88b052e service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:22.481927 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-68de106c-a4b4-429a-8957-cd0974ddfee7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1eaf427c6044461a8000994e2f4832c', 'request_id': 'req-68de106c-a4b4-429a-8957-cd0974ddfee7'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:22.516407 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-68de106c-a4b4-429a-8957-cd0974ddfee7 admin admin] The token cache is being invalidate because project f1eaf427c6044461a8000994e2f4832c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:05:22.516705 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-68de106c-a4b4-429a-8957-cd0974ddfee7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:22.518016 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-68de106c-a4b4-429a-8957-cd0974ddfee7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:22.522501 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 821/1643] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:05:22 2022] DELETE /identity/v3/projects/f1eaf427c6044461a8000994e2f4832c => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:22.530495 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc23593-d9ed-463c-84c6-2786b0df261a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:22.530782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc23593-d9ed-463c-84c6-2786b0df261a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:22.531066 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc23593-d9ed-463c-84c6-2786b0df261a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:22.537759 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a575d-b3bf-4be3-ac7b-c4fc196a507c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:22.542598 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a575d-b3bf-4be3-ac7b-c4fc196a507c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:22.542598 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a575d-b3bf-4be3-ac7b-c4fc196a507c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:22.542598 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a575d-b3bf-4be3-ac7b-c4fc196a507c admin admin] PATH_INFO: `/v3/users/066eeca5e1e64725ab8081c34121b3ce` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:22.542598 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a575d-b3bf-4be3-ac7b-c4fc196a507c admin admin] RBAC: Authorizing `identity:delete_user(user_id=066eeca5e1e64725ab8081c34121b3ce)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:22.546277 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a575d-b3bf-4be3-ac7b-c4fc196a507c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:22.600383 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-0dc23593-d9ed-463c-84c6-2786b0df261a None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:22.604995 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0dc23593-d9ed-463c-84c6-2786b0df261a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:22.620030 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 822/1644] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:22.621298 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc23593-d9ed-463c-84c6-2786b0df261a None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:22.622737 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0dc23593-d9ed-463c-84c6-2786b0df261a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:22.650350 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc23593-d9ed-463c-84c6-2786b0df261a None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:22.663214 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a80db34-dc81-41d0-92c6-fc2814f61a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:22.665454 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80db34-dc81-41d0-92c6-fc2814f61a93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:22.665454 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80db34-dc81-41d0-92c6-fc2814f61a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:22.665454 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80db34-dc81-41d0-92c6-fc2814f61a93 admin admin] PATH_INFO: `/v3/projects/565a50447ba547cb871ff2365b635131` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:22.669569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a80db34-dc81-41d0-92c6-fc2814f61a93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=565a50447ba547cb871ff2365b635131)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:22.672138 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a80db34-dc81-41d0-92c6-fc2814f61a93 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:22.798575 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-315a575d-b3bf-4be3-ac7b-c4fc196a507c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '066eeca5e1e64725ab8081c34121b3ce'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:22.799401 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-315a575d-b3bf-4be3-ac7b-c4fc196a507c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '066eeca5e1e64725ab8081c34121b3ce'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:22.823220 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-315a575d-b3bf-4be3-ac7b-c4fc196a507c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '066eeca5e1e64725ab8081c34121b3ce'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:22.830098 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:22.842700 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4a80db34-dc81-41d0-92c6-fc2814f61a93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '565a50447ba547cb871ff2365b635131', 'request_id': 'req-4a80db34-dc81-41d0-92c6-fc2814f61a93'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:22.850844 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-315a575d-b3bf-4be3-ac7b-c4fc196a507c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '066eeca5e1e64725ab8081c34121b3ce'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:22.851659 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-315a575d-b3bf-4be3-ac7b-c4fc196a507c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '066eeca5e1e64725ab8081c34121b3ce'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:22.859530 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4a80db34-dc81-41d0-92c6-fc2814f61a93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '565a50447ba547cb871ff2365b635131', 'request_id': 'req-4a80db34-dc81-41d0-92c6-fc2814f61a93'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:22.917495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 823/1645] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:05:22 2022] DELETE /identity/v3/users/066eeca5e1e64725ab8081c34121b3ce => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:05:22.917751 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ef36e-8c49-422d-9378-ea448940c994 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:22.918273 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ef36e-8c49-422d-9378-ea448940c994 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:22.918665 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ef36e-8c49-422d-9378-ea448940c994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:22.961326 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-888ef36e-8c49-422d-9378-ea448940c994 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:22.964042 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-888ef36e-8c49-422d-9378-ea448940c994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:22.994439 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4a80db34-dc81-41d0-92c6-fc2814f61a93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '565a50447ba547cb871ff2365b635131', 'request_id': 'req-4a80db34-dc81-41d0-92c6-fc2814f61a93'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:23.016194 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4a80db34-dc81-41d0-92c6-fc2814f61a93 admin admin] The token cache is being invalidate because project 565a50447ba547cb871ff2365b635131 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:05:23.016194 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4a80db34-dc81-41d0-92c6-fc2814f61a93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:23.017042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4a80db34-dc81-41d0-92c6-fc2814f61a93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:23.024009 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 823/1646] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:05:22 2022] DELETE /identity/v3/projects/565a50447ba547cb871ff2365b635131 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:23.032440 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e87af4-0ae1-4865-bc23-e2a8a1772b58 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:23.032440 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e87af4-0ae1-4865-bc23-e2a8a1772b58 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:23.032440 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e87af4-0ae1-4865-bc23-e2a8a1772b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:23.092658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-27e87af4-0ae1-4865-bc23-e2a8a1772b58 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:23.094806 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-27e87af4-0ae1-4865-bc23-e2a8a1772b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:23.144636 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 824/1647] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:23.145723 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ef36e-8c49-422d-9378-ea448940c994 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:23.148322 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-888ef36e-8c49-422d-9378-ea448940c994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:23.190270 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ef36e-8c49-422d-9378-ea448940c994 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:23.209286 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c04227-0d94-4774-9644-816f9b9fe48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:23.209777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c04227-0d94-4774-9644-816f9b9fe48d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:23.210130 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c04227-0d94-4774-9644-816f9b9fe48d admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:23.210474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c04227-0d94-4774-9644-816f9b9fe48d admin admin] PATH_INFO: `/v3/users/27d52d2e932b42ca97ea52289186d1a1` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:23.211920 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c04227-0d94-4774-9644-816f9b9fe48d admin admin] RBAC: Authorizing `identity:delete_user(user_id=27d52d2e932b42ca97ea52289186d1a1)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:23.214008 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c04227-0d94-4774-9644-816f9b9fe48d admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:23.269988 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 824/1648] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:23.275397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e87af4-0ae1-4865-bc23-e2a8a1772b58 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:23.291431 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e87af4-0ae1-4865-bc23-e2a8a1772b58 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:23.297457 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a6f3b9-adae-4833-9807-9a235b8b99e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:23.298666 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6f3b9-adae-4833-9807-9a235b8b99e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:23.299095 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6f3b9-adae-4833-9807-9a235b8b99e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:23.299499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6f3b9-adae-4833-9807-9a235b8b99e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:23.300642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a6f3b9-adae-4833-9807-9a235b8b99e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:05:23.301110 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a6f3b9-adae-4833-9807-9a235b8b99e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:23.303262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a6f3b9-adae-4833-9807-9a235b8b99e6 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:23.320900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 825/1649] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:05:23.323556 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a6f3b9-adae-4833-9807-9a235b8b99e6 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:23.366482 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a6f3b9-adae-4833-9807-9a235b8b99e6 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:23.381889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b445b7b1-79b3-4f60-8993-a5bcd39793cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:23.383097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445b7b1-79b3-4f60-8993-a5bcd39793cc admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:23.383386 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445b7b1-79b3-4f60-8993-a5bcd39793cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:23.383887 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445b7b1-79b3-4f60-8993-a5bcd39793cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:23.385530 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b445b7b1-79b3-4f60-8993-a5bcd39793cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:23.389455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b445b7b1-79b3-4f60-8993-a5bcd39793cc admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:23.543448 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-49c04227-0d94-4774-9644-816f9b9fe48d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27d52d2e932b42ca97ea52289186d1a1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:23.545483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-49c04227-0d94-4774-9644-816f9b9fe48d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27d52d2e932b42ca97ea52289186d1a1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:23.556203 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-49c04227-0d94-4774-9644-816f9b9fe48d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27d52d2e932b42ca97ea52289186d1a1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:23.566109 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 826/1650] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:05:23 2022] POST /identity/v3/projects => generated 379 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:23.571807 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b445b7b1-79b3-4f60-8993-a5bcd39793cc admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:23.574068 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b445b7b1-79b3-4f60-8993-a5bcd39793cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:23.583549 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-49c04227-0d94-4774-9644-816f9b9fe48d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27d52d2e932b42ca97ea52289186d1a1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:23.584670 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-49c04227-0d94-4774-9644-816f9b9fe48d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27d52d2e932b42ca97ea52289186d1a1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:23.703761 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 825/1651] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:05:23 2022] DELETE /identity/v3/users/27d52d2e932b42ca97ea52289186d1a1 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:23.712128 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c04227-0d94-4774-9644-816f9b9fe48d admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:23.720854 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-49c04227-0d94-4774-9644-816f9b9fe48d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:23.749868 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b445b7b1-79b3-4f60-8993-a5bcd39793cc admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:23.960332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90293e30-10fa-4c91-8f1b-fec577e134c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:23.960332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90293e30-10fa-4c91-8f1b-fec577e134c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:23.960332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90293e30-10fa-4c91-8f1b-fec577e134c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:23.960332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90293e30-10fa-4c91-8f1b-fec577e134c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:23.966439 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-90293e30-10fa-4c91-8f1b-fec577e134c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:24.005409 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c04227-0d94-4774-9644-816f9b9fe48d admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:24.018699 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b618d8-c58e-49b5-ad79-5e3b70ca9d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:24.019431 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b618d8-c58e-49b5-ad79-5e3b70ca9d87 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:24.019721 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b618d8-c58e-49b5-ad79-5e3b70ca9d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:24.020000 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b618d8-c58e-49b5-ad79-5e3b70ca9d87 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:24.020695 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b618d8-c58e-49b5-ad79-5e3b70ca9d87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:24.027797 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b618d8-c58e-49b5-ad79-5e3b70ca9d87 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:24.036978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90293e30-10fa-4c91-8f1b-fec577e134c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:24.042459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90293e30-10fa-4c91-8f1b-fec577e134c1 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:24.075981 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 827/1652] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:24.169222 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 826/1653] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:05:23 2022] POST /identity/v3/users => generated 290 bytes in 465 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:24.174691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90293e30-10fa-4c91-8f1b-fec577e134c1 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:24.195933 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90293e30-10fa-4c91-8f1b-fec577e134c1 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:24.205694 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb0545-afad-4dbe-a253-cc531462c2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:24.207115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb0545-afad-4dbe-a253-cc531462c2e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:24.207557 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb0545-afad-4dbe-a253-cc531462c2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:24.208046 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb0545-afad-4dbe-a253-cc531462c2e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:24.209384 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb0545-afad-4dbe-a253-cc531462c2e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:24.212404 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb0545-afad-4dbe-a253-cc531462c2e9 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:24.241108 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 828/1654] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:24.241231 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b618d8-c58e-49b5-ad79-5e3b70ca9d87 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:24.265034 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b618d8-c58e-49b5-ad79-5e3b70ca9d87 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:24.281208 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3730dd2-64e3-4126-912c-1d2cbee02e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:24.282236 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3730dd2-64e3-4126-912c-1d2cbee02e48 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:24.282728 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3730dd2-64e3-4126-912c-1d2cbee02e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:24.283196 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3730dd2-64e3-4126-912c-1d2cbee02e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:24.284262 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3730dd2-64e3-4126-912c-1d2cbee02e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:24.290282 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3730dd2-64e3-4126-912c-1d2cbee02e48 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:24.316207 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb0545-afad-4dbe-a253-cc531462c2e9 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:24.334806 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 827/1655] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:24.341098 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb0545-afad-4dbe-a253-cc531462c2e9 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:24.347821 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073f77f-ecc4-451e-8646-30d1ee8b213b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:24.349011 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073f77f-ecc4-451e-8646-30d1ee8b213b admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:24.350405 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073f77f-ecc4-451e-8646-30d1ee8b213b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:24.350884 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073f77f-ecc4-451e-8646-30d1ee8b213b admin admin] PATH_INFO: `/v3/projects/5354a22bd97e424f8acb938816236ff3/users/226b35df09724aaca8c906df71f3bc60/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:24.375465 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c073f77f-ecc4-451e-8646-30d1ee8b213b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5354a22bd97e424f8acb938816236ff3, user_id=226b35df09724aaca8c906df71f3bc60, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:24.385375 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c073f77f-ecc4-451e-8646-30d1ee8b213b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:24.484967 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 829/1656] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:24 2022] PUT /identity/v3/projects/5354a22bd97e424f8acb938816236ff3/users/226b35df09724aaca8c906df71f3bc60/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:24.497913 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d318f-8656-4168-ac7e-2d0716d43f81 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:24.498754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d318f-8656-4168-ac7e-2d0716d43f81 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:24.499354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d318f-8656-4168-ac7e-2d0716d43f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:24.570922 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-284d318f-8656-4168-ac7e-2d0716d43f81 None None] MFA Rules not processed for user `226b35df09724aaca8c906df71f3bc60`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:24.578828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-284d318f-8656-4168-ac7e-2d0716d43f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:24.642744 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:24.841757 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:24.870849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073f77f-ecc4-451e-8646-30d1ee8b213b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:24.882982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 828/1657] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:25.058365 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073f77f-ecc4-451e-8646-30d1ee8b213b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:25.068622 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa5cf6-11d6-4a54-9bd0-df81fec2a02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:25.070718 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa5cf6-11d6-4a54-9bd0-df81fec2a02f admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:25.072647 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa5cf6-11d6-4a54-9bd0-df81fec2a02f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:25.073694 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa5cf6-11d6-4a54-9bd0-df81fec2a02f admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:25.074617 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa5cf6-11d6-4a54-9bd0-df81fec2a02f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:25.078125 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa5cf6-11d6-4a54-9bd0-df81fec2a02f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:25.113332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d318f-8656-4168-ac7e-2d0716d43f81 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:25.114823 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-284d318f-8656-4168-ac7e-2d0716d43f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:25.154300 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d318f-8656-4168-ac7e-2d0716d43f81 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:25.166487 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc196db-5c9f-4c4a-afbe-5c5d0ad802ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:25.167821 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc196db-5c9f-4c4a-afbe-5c5d0ad802ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:25.168412 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc196db-5c9f-4c4a-afbe-5c5d0ad802ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:25.168789 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc196db-5c9f-4c4a-afbe-5c5d0ad802ef admin admin] PATH_INFO: `/v3/projects/82df673d62fb411e9b0f9725f173c3f6` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:25.171827 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc196db-5c9f-4c4a-afbe-5c5d0ad802ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=82df673d62fb411e9b0f9725f173c3f6)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:25.173846 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc196db-5c9f-4c4a-afbe-5c5d0ad802ef admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:25.240319 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 830/1658] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:05:24 2022] POST /identity/v3/projects => generated 379 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:25.245310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa5cf6-11d6-4a54-9bd0-df81fec2a02f admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:25.381717 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1cc196db-5c9f-4c4a-afbe-5c5d0ad802ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82df673d62fb411e9b0f9725f173c3f6', 'request_id': 'req-1cc196db-5c9f-4c4a-afbe-5c5d0ad802ef'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:25.382032 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa5cf6-11d6-4a54-9bd0-df81fec2a02f admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:25.487509 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9d151-feab-4a5f-b8bb-6f3fbabe3a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:25.488326 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9d151-feab-4a5f-b8bb-6f3fbabe3a81 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:25.488707 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9d151-feab-4a5f-b8bb-6f3fbabe3a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:25.489212 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9d151-feab-4a5f-b8bb-6f3fbabe3a81 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:25.490194 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe9d151-feab-4a5f-b8bb-6f3fbabe3a81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:25.492847 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe9d151-feab-4a5f-b8bb-6f3fbabe3a81 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:25.508772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1cc196db-5c9f-4c4a-afbe-5c5d0ad802ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82df673d62fb411e9b0f9725f173c3f6', 'request_id': 'req-1cc196db-5c9f-4c4a-afbe-5c5d0ad802ef'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:25.520539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1cc196db-5c9f-4c4a-afbe-5c5d0ad802ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82df673d62fb411e9b0f9725f173c3f6', 'request_id': 'req-1cc196db-5c9f-4c4a-afbe-5c5d0ad802ef'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:25.535026 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1cc196db-5c9f-4c4a-afbe-5c5d0ad802ef admin admin] The token cache is being invalidate because project 82df673d62fb411e9b0f9725f173c3f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:05:25.536131 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1cc196db-5c9f-4c4a-afbe-5c5d0ad802ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:25.537333 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1cc196db-5c9f-4c4a-afbe-5c5d0ad802ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:25.540387 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 829/1659] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:05:25 2022] DELETE /identity/v3/projects/82df673d62fb411e9b0f9725f173c3f6 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:25.545457 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51073c53-6480-48ca-87c6-2edbc9b49d78 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:25.553345 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51073c53-6480-48ca-87c6-2edbc9b49d78 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:25.553345 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51073c53-6480-48ca-87c6-2edbc9b49d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:25.585286 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 831/1660] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:05:25 2022] POST /identity/v3/users => generated 296 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:05:25.596841 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda8843-77d1-4d5e-b7fd-d36b2a1c4c3b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:25.600193 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda8843-77d1-4d5e-b7fd-d36b2a1c4c3b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:25.601201 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda8843-77d1-4d5e-b7fd-d36b2a1c4c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:25.611140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-51073c53-6480-48ca-87c6-2edbc9b49d78 None None] MFA Rules not processed for user `c764d9778ef74b709cedbccabac37af0`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:25.612560 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-51073c53-6480-48ca-87c6-2edbc9b49d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:25.683558 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-acda8843-77d1-4d5e-b7fd-d36b2a1c4c3b None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:25.690356 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-acda8843-77d1-4d5e-b7fd-d36b2a1c4c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:25.704600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 832/1661] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:25.707186 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda8843-77d1-4d5e-b7fd-d36b2a1c4c3b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:25.708983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-acda8843-77d1-4d5e-b7fd-d36b2a1c4c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:25.747553 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda8843-77d1-4d5e-b7fd-d36b2a1c4c3b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:25.752089 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 830/1662] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:25 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:25.754268 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb828a-138e-499d-a50e-25ac7698ddf2 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:25.754714 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb828a-138e-499d-a50e-25ac7698ddf2 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:25.755112 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb828a-138e-499d-a50e-25ac7698ddf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:25.765826 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68d744-ce9d-463f-89f7-c82d89fe76dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:25.766456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68d744-ce9d-463f-89f7-c82d89fe76dd admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:25.766966 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68d744-ce9d-463f-89f7-c82d89fe76dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:25.767714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68d744-ce9d-463f-89f7-c82d89fe76dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:25.768870 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68d744-ce9d-463f-89f7-c82d89fe76dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:25.770956 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68d744-ce9d-463f-89f7-c82d89fe76dd admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:25.790211 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 833/1663] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:25.790211 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01617df-b0cd-4542-85eb-050858e6683f None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:25.790211 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01617df-b0cd-4542-85eb-050858e6683f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:25.790211 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01617df-b0cd-4542-85eb-050858e6683f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:25.805122 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-1dbb828a-138e-499d-a50e-25ac7698ddf2 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:25.807443 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-1dbb828a-138e-499d-a50e-25ac7698ddf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:25.812582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a01617df-b0cd-4542-85eb-050858e6683f None None] MFA Rules not processed for user `c764d9778ef74b709cedbccabac37af0`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:25.813792 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a01617df-b0cd-4542-85eb-050858e6683f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:25.817671 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 831/1664] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:25.818944 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb828a-138e-499d-a50e-25ac7698ddf2 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:25.827680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 834/1665] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:25 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:25.827680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01617df-b0cd-4542-85eb-050858e6683f None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:25.840668 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb828a-138e-499d-a50e-25ac7698ddf2 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:25.853456 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89737560-2652-47c2-bcc8-36799bd08fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:25.854146 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89737560-2652-47c2-bcc8-36799bd08fad admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:25.854429 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89737560-2652-47c2-bcc8-36799bd08fad admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:25.854689 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89737560-2652-47c2-bcc8-36799bd08fad admin admin] PATH_INFO: `/v3/projects/3b9a776450c4463ba8648c8f1a33fdb3/users/58154460e86d4f4f9b11af2beb3f828a/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:25.862472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01617df-b0cd-4542-85eb-050858e6683f None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:25.868610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc665b-4d4f-4e64-a861-4298811f22e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:25.869281 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc665b-4d4f-4e64-a861-4298811f22e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:25.869566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc665b-4d4f-4e64-a861-4298811f22e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:25.869839 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc665b-4d4f-4e64-a861-4298811f22e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:25.870645 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc665b-4d4f-4e64-a861-4298811f22e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:05:25.870956 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc665b-4d4f-4e64-a861-4298811f22e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:25.872814 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc665b-4d4f-4e64-a861-4298811f22e8 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:25.900505 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89737560-2652-47c2-bcc8-36799bd08fad admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b9a776450c4463ba8648c8f1a33fdb3, user_id=58154460e86d4f4f9b11af2beb3f828a, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:25.900766 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 835/1666] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:05:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:05:25.900850 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89737560-2652-47c2-bcc8-36799bd08fad admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:25.901025 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc665b-4d4f-4e64-a861-4298811f22e8 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:25.901025 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0bbc665b-4d4f-4e64-a861-4298811f22e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:25.913527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc665b-4d4f-4e64-a861-4298811f22e8 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:25.920195 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf20bc4d-a2c5-44a3-8af3-5f0b84b8bafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:25.920925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20bc4d-a2c5-44a3-8af3-5f0b84b8bafe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:25.921274 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20bc4d-a2c5-44a3-8af3-5f0b84b8bafe admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:25.921555 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20bc4d-a2c5-44a3-8af3-5f0b84b8bafe admin admin] PATH_INFO: `/v3/projects/4e7003778bfe41bbb0571fa23db64b7a` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:25.922901 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf20bc4d-a2c5-44a3-8af3-5f0b84b8bafe admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e7003778bfe41bbb0571fa23db64b7a)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:25.925145 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf20bc4d-a2c5-44a3-8af3-5f0b84b8bafe admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:25.989868 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 832/1667] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:25 2022] PUT /identity/v3/projects/3b9a776450c4463ba8648c8f1a33fdb3/users/58154460e86d4f4f9b11af2beb3f828a/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:25.990841 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89737560-2652-47c2-bcc8-36799bd08fad admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:25.996556 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-89737560-2652-47c2-bcc8-36799bd08fad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:26.084081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bf20bc4d-a2c5-44a3-8af3-5f0b84b8bafe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e7003778bfe41bbb0571fa23db64b7a', 'request_id': 'req-bf20bc4d-a2c5-44a3-8af3-5f0b84b8bafe'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:26.095910 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bf20bc4d-a2c5-44a3-8af3-5f0b84b8bafe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e7003778bfe41bbb0571fa23db64b7a', 'request_id': 'req-bf20bc4d-a2c5-44a3-8af3-5f0b84b8bafe'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:26.178526 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bf20bc4d-a2c5-44a3-8af3-5f0b84b8bafe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e7003778bfe41bbb0571fa23db64b7a', 'request_id': 'req-bf20bc4d-a2c5-44a3-8af3-5f0b84b8bafe'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:26.185151 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89737560-2652-47c2-bcc8-36799bd08fad admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:26.196773 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bf20bc4d-a2c5-44a3-8af3-5f0b84b8bafe admin admin] The token cache is being invalidate because project 4e7003778bfe41bbb0571fa23db64b7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:05:26.199624 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bf20bc4d-a2c5-44a3-8af3-5f0b84b8bafe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:26.200694 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bf20bc4d-a2c5-44a3-8af3-5f0b84b8bafe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:26.206145 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 836/1668] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:05:25 2022] DELETE /identity/v3/projects/4e7003778bfe41bbb0571fa23db64b7a => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:26.213122 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf20bc4d-a2c5-44a3-8af3-5f0b84b8bafe admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:26.215026 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bf20bc4d-a2c5-44a3-8af3-5f0b84b8bafe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:26.270352 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:26.341607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dfb248-0fd6-4d98-8b5a-33517f62661a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:26.342256 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfb248-0fd6-4d98-8b5a-33517f62661a service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:26.342564 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfb248-0fd6-4d98-8b5a-33517f62661a service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:26.342867 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfb248-0fd6-4d98-8b5a-33517f62661a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:26.345708 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f4dfb248-0fd6-4d98-8b5a-33517f62661a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:26.357016 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:26.391865 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf20bc4d-a2c5-44a3-8af3-5f0b84b8bafe admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:26.397177 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ca3dc-0aab-4581-bc2d-2de181875dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:26.397886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ca3dc-0aab-4581-bc2d-2de181875dde admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:26.398283 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ca3dc-0aab-4581-bc2d-2de181875dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:26.398475 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ca3dc-0aab-4581-bc2d-2de181875dde admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:26.399147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ca3dc-0aab-4581-bc2d-2de181875dde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:26.401823 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ca3dc-0aab-4581-bc2d-2de181875dde admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:26.483733 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dfb248-0fd6-4d98-8b5a-33517f62661a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:26.486691 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dfb248-0fd6-4d98-8b5a-33517f62661a service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:26.504795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 833/1669] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:25 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:26.506528 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dfb248-0fd6-4d98-8b5a-33517f62661a service nova] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:26.508849 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f4dfb248-0fd6-4d98-8b5a-33517f62661a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:26.522457 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 837/1670] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:05:26 2022] POST /identity/v3/projects => generated 365 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:26.524098 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830a029-5f4c-4ab2-a2e3-541603f12e7b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:26.524404 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830a029-5f4c-4ab2-a2e3-541603f12e7b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:26.526284 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830a029-5f4c-4ab2-a2e3-541603f12e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:26.530278 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dfb248-0fd6-4d98-8b5a-33517f62661a service nova] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:26.577027 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-7830a029-5f4c-4ab2-a2e3-541603f12e7b None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:26.578883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7830a029-5f4c-4ab2-a2e3-541603f12e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:26.611887 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ed8a0-14a7-4ec3-825d-873c719ceaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:26.613336 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ed8a0-14a7-4ec3-825d-873c719ceaa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:26.613771 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ed8a0-14a7-4ec3-825d-873c719ceaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:26.614185 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ed8a0-14a7-4ec3-825d-873c719ceaa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:26.614986 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ed8a0-14a7-4ec3-825d-873c719ceaa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:26.617460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ed8a0-14a7-4ec3-825d-873c719ceaa1 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:26.641021 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 834/1671] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:26 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:26.641962 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ed8a0-14a7-4ec3-825d-873c719ceaa1 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:26.667491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ed8a0-14a7-4ec3-825d-873c719ceaa1 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:26.677326 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a17a3-e17f-4215-8515-347f5bb91d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:26.677845 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a17a3-e17f-4215-8515-347f5bb91d3c admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:26.680378 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a17a3-e17f-4215-8515-347f5bb91d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:26.680703 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a17a3-e17f-4215-8515-347f5bb91d3c admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:26.681585 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a17a3-e17f-4215-8515-347f5bb91d3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:26.686250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a17a3-e17f-4215-8515-347f5bb91d3c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:26.719818 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 838/1672] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:26.719818 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7830a029-5f4c-4ab2-a2e3-541603f12e7b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:26.729712 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7830a029-5f4c-4ab2-a2e3-541603f12e7b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:26.747777 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb554dfe-4c49-4760-9ba9-316c6452587d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:26.748643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb554dfe-4c49-4760-9ba9-316c6452587d admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:26.749070 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb554dfe-4c49-4760-9ba9-316c6452587d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:26.749354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb554dfe-4c49-4760-9ba9-316c6452587d admin admin] PATH_INFO: `/v3/projects/3b9a776450c4463ba8648c8f1a33fdb3/users/58154460e86d4f4f9b11af2beb3f828a/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:26.754703 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb554dfe-4c49-4760-9ba9-316c6452587d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b9a776450c4463ba8648c8f1a33fdb3, user_id=58154460e86d4f4f9b11af2beb3f828a, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:26.761675 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb554dfe-4c49-4760-9ba9-316c6452587d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:26.786989 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 839/1673] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:26 2022] PUT /identity/v3/projects/3b9a776450c4463ba8648c8f1a33fdb3/users/58154460e86d4f4f9b11af2beb3f828a/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:26.798906 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77a61d-cad3-43ff-b343-73127575224b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:26.799197 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77a61d-cad3-43ff-b343-73127575224b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:26.799473 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77a61d-cad3-43ff-b343-73127575224b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:26.810157 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 835/1674] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:05:26 2022] POST /identity/v3/users => generated 283 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:26.812308 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33211fca-1543-424c-b260-be69fe7f6cf5 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:26.812774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33211fca-1543-424c-b260-be69fe7f6cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:26.813221 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33211fca-1543-424c-b260-be69fe7f6cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:26.834314 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-dc77a61d-cad3-43ff-b343-73127575224b None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:26.836581 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-dc77a61d-cad3-43ff-b343-73127575224b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:26.851627 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-33211fca-1543-424c-b260-be69fe7f6cf5 None None] MFA Rules not processed for user `58154460e86d4f4f9b11af2beb3f828a`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:26.853481 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-33211fca-1543-424c-b260-be69fe7f6cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:26.991140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 836/1675] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:26 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:26.993477 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33211fca-1543-424c-b260-be69fe7f6cf5 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:27.000010 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 840/1676] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:27.003120 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77a61d-cad3-43ff-b343-73127575224b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:27.010882 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-dc77a61d-cad3-43ff-b343-73127575224b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:27.023970 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33211fca-1543-424c-b260-be69fe7f6cf5 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:27.036339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769460c-a42d-4c3f-8461-5cadc7623de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:27.038076 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769460c-a42d-4c3f-8461-5cadc7623de4 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:27.041103 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769460c-a42d-4c3f-8461-5cadc7623de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:27.041913 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769460c-a42d-4c3f-8461-5cadc7623de4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:27.042778 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769460c-a42d-4c3f-8461-5cadc7623de4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:27.048365 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769460c-a42d-4c3f-8461-5cadc7623de4 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:27.075559 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 837/1677] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:26 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:27.078322 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769460c-a42d-4c3f-8461-5cadc7623de4 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:27.099642 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769460c-a42d-4c3f-8461-5cadc7623de4 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:27.110959 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18f1ac-f48c-4fe2-ac3d-8fcdbcada762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:27.111586 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18f1ac-f48c-4fe2-ac3d-8fcdbcada762 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:27.111988 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18f1ac-f48c-4fe2-ac3d-8fcdbcada762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:27.112197 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18f1ac-f48c-4fe2-ac3d-8fcdbcada762 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:27.112926 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe18f1ac-f48c-4fe2-ac3d-8fcdbcada762 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:05:27.113164 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe18f1ac-f48c-4fe2-ac3d-8fcdbcada762 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:27.114772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe18f1ac-f48c-4fe2-ac3d-8fcdbcada762 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:27.130207 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77a61d-cad3-43ff-b343-73127575224b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:27.135506 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122a060-c307-4254-b8a0-118694680bf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:27.137233 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122a060-c307-4254-b8a0-118694680bf2 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:27.137619 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122a060-c307-4254-b8a0-118694680bf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:27.137998 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122a060-c307-4254-b8a0-118694680bf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:27.152673 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 838/1678] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:05:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:05:27.153697 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18f1ac-f48c-4fe2-ac3d-8fcdbcada762 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:27.170958 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e122a060-c307-4254-b8a0-118694680bf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:27.177285 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e122a060-c307-4254-b8a0-118694680bf2 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:27.177470 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18f1ac-f48c-4fe2-ac3d-8fcdbcada762 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:27.189076 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0d6a2b-42d7-46da-b726-84c68d4d370d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:27.189076 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d6a2b-42d7-46da-b726-84c68d4d370d admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:27.189076 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d6a2b-42d7-46da-b726-84c68d4d370d admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:27.189076 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d6a2b-42d7-46da-b726-84c68d4d370d admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:27.189547 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0d6a2b-42d7-46da-b726-84c68d4d370d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:27.191303 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0d6a2b-42d7-46da-b726-84c68d4d370d admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:27.208995 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 841/1679] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:27 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:27.209755 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122a060-c307-4254-b8a0-118694680bf2 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:27.230921 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122a060-c307-4254-b8a0-118694680bf2 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:27.233013 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 839/1680] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:27.237494 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0d6a2b-42d7-46da-b726-84c68d4d370d admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:27.241025 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c4f540-9469-429b-85f4-1bfbd934bf51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:27.242491 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4f540-9469-429b-85f4-1bfbd934bf51 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:27.242915 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4f540-9469-429b-85f4-1bfbd934bf51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:27.243327 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4f540-9469-429b-85f4-1bfbd934bf51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:27.249838 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c4f540-9469-429b-85f4-1bfbd934bf51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:27.253652 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c4f540-9469-429b-85f4-1bfbd934bf51 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:27.265873 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0d6a2b-42d7-46da-b726-84c68d4d370d admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:27.268484 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:27.270127 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86428daa-4cd0-4f67-8522-435f49cf5ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:27.270815 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86428daa-4cd0-4f67-8522-435f49cf5ce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:27.271099 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86428daa-4cd0-4f67-8522-435f49cf5ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:27.271375 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86428daa-4cd0-4f67-8522-435f49cf5ce7 admin admin] PATH_INFO: `/v3/projects/ad067e2527ff4e7d9bed04c66c91c1df/users/33776df0eed64572b88df64e500207d7/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:27.306300 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86428daa-4cd0-4f67-8522-435f49cf5ce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad067e2527ff4e7d9bed04c66c91c1df, user_id=33776df0eed64572b88df64e500207d7, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:27.311975 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86428daa-4cd0-4f67-8522-435f49cf5ce7 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:27.405069 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 842/1681] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:05:27 2022] POST /identity/v3/projects => generated 369 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:27.406874 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbd86c-6861-41cb-9778-0498978b16eb None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:27.407463 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbd86c-6861-41cb-9778-0498978b16eb None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:27.407807 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbd86c-6861-41cb-9778-0498978b16eb None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:27.409192 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 843/1682] 10.222.0.48 () {54 vars in 912 bytes} [Fri Sep 9 21:05:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:05:27.410000 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbbd86c-6861-41cb-9778-0498978b16eb None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:27.467837 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 840/1683] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:27 2022] PUT /identity/v3/projects/ad067e2527ff4e7d9bed04c66c91c1df/users/33776df0eed64572b88df64e500207d7/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:27.467837 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86aac73-780f-4eab-b450-8ff0982f1b77 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:27.467837 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86aac73-780f-4eab-b450-8ff0982f1b77 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:27.467837 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86aac73-780f-4eab-b450-8ff0982f1b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:27.497597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbbd86c-6861-41cb-9778-0498978b16eb None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:27.595842 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-c86aac73-780f-4eab-b450-8ff0982f1b77 None None] MFA Rules not processed for user `8f5dfac06ac9492a86503ba133c638a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:27.599389 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c86aac73-780f-4eab-b450-8ff0982f1b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:27.605063 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21992fb7-3300-423f-9d32-f8b4fae3476c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:27.606043 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21992fb7-3300-423f-9d32-f8b4fae3476c admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:27.607105 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21992fb7-3300-423f-9d32-f8b4fae3476c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:27.607399 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21992fb7-3300-423f-9d32-f8b4fae3476c admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:27.610019 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21992fb7-3300-423f-9d32-f8b4fae3476c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:27.618270 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21992fb7-3300-423f-9d32-f8b4fae3476c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:27.733850 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 841/1684] 10.222.0.48 () {58 vars in 1009 bytes} [Fri Sep 9 21:05:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:27.738407 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba550388-3922-4957-99b1-83b3cb899ac6 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:27.738852 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba550388-3922-4957-99b1-83b3cb899ac6 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:27.739282 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba550388-3922-4957-99b1-83b3cb899ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:27.806325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 844/1685] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:05:27 2022] POST /identity/v3/users => generated 285 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:27.807218 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21992fb7-3300-423f-9d32-f8b4fae3476c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:27.816141 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-21992fb7-3300-423f-9d32-f8b4fae3476c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:27.839364 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-ba550388-3922-4957-99b1-83b3cb899ac6 None None] MFA Rules not processed for user `33776df0eed64572b88df64e500207d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:27.845639 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ba550388-3922-4957-99b1-83b3cb899ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:27.873784 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21992fb7-3300-423f-9d32-f8b4fae3476c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:27.888571 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946533c3-d418-4d0e-846a-1ed10d42d005 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:27.889448 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946533c3-d418-4d0e-846a-1ed10d42d005 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:27.890455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946533c3-d418-4d0e-846a-1ed10d42d005 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:27.892149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946533c3-d418-4d0e-846a-1ed10d42d005 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:27.928750 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946533c3-d418-4d0e-846a-1ed10d42d005 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:27.930704 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946533c3-d418-4d0e-846a-1ed10d42d005 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:27.955132 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 845/1686] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:27.956246 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946533c3-d418-4d0e-846a-1ed10d42d005 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:27.981498 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946533c3-d418-4d0e-846a-1ed10d42d005 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:27.993510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba941d0-2adc-41bd-8d6f-387f1b98e333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:27.994245 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba941d0-2adc-41bd-8d6f-387f1b98e333 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:27.997379 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba941d0-2adc-41bd-8d6f-387f1b98e333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:27.997379 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba941d0-2adc-41bd-8d6f-387f1b98e333 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:27.997379 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba941d0-2adc-41bd-8d6f-387f1b98e333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:27.998689 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba941d0-2adc-41bd-8d6f-387f1b98e333 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:28.006455 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 842/1687] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:28.014757 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba550388-3922-4957-99b1-83b3cb899ac6 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:28.036993 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba550388-3922-4957-99b1-83b3cb899ac6 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:28.044823 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 846/1688] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:28.049124 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba941d0-2adc-41bd-8d6f-387f1b98e333 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:28.060645 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba941d0-2adc-41bd-8d6f-387f1b98e333 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:28.062895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4f9db-6094-4c68-bf7d-1408d9556797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:28.063629 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4f9db-6094-4c68-bf7d-1408d9556797 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:28.064001 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4f9db-6094-4c68-bf7d-1408d9556797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:28.064304 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4f9db-6094-4c68-bf7d-1408d9556797 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:28.065180 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f4f9db-6094-4c68-bf7d-1408d9556797 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:28.066522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7096b-1e83-4f94-bd05-0ac949ace800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:28.067213 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f4f9db-6094-4c68-bf7d-1408d9556797 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:28.067434 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7096b-1e83-4f94-bd05-0ac949ace800 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:28.067969 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7096b-1e83-4f94-bd05-0ac949ace800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:28.068626 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7096b-1e83-4f94-bd05-0ac949ace800 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:28.069376 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb7096b-1e83-4f94-bd05-0ac949ace800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:28.071753 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb7096b-1e83-4f94-bd05-0ac949ace800 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:28.091744 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 847/1689] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:28.093089 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7096b-1e83-4f94-bd05-0ac949ace800 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:28.105001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7096b-1e83-4f94-bd05-0ac949ace800 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:28.111168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9f9ea-7b5e-4be7-890e-968f605a41ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:28.111972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9f9ea-7b5e-4be7-890e-968f605a41ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:28.112413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9f9ea-7b5e-4be7-890e-968f605a41ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:28.113144 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9f9ea-7b5e-4be7-890e-968f605a41ad admin admin] PATH_INFO: `/v3/projects/24f926d61a1f434691d24e5445d60dda/users/d3cb3cdb7c56482a86df55b0d85b4d0f/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:28.132045 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa9f9ea-7b5e-4be7-890e-968f605a41ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=24f926d61a1f434691d24e5445d60dda, user_id=d3cb3cdb7c56482a86df55b0d85b4d0f, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:28.136341 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa9f9ea-7b5e-4be7-890e-968f605a41ad admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:28.222600 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:28.229268 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 843/1690] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:05:28 2022] POST /identity/v3/projects => generated 363 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:28.233788 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4f9db-6094-4c68-bf7d-1408d9556797 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:28.284843 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 848/1691] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:28 2022] PUT /identity/v3/projects/24f926d61a1f434691d24e5445d60dda/users/d3cb3cdb7c56482a86df55b0d85b4d0f/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:28.288896 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2ac77-ab79-4797-8253-d3affaf96243 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:28.289203 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2ac77-ab79-4797-8253-d3affaf96243 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:28.289465 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2ac77-ab79-4797-8253-d3affaf96243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:28.374512 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-acb2ac77-ab79-4797-8253-d3affaf96243 None None] MFA Rules not processed for user `d3cb3cdb7c56482a86df55b0d85b4d0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:28.376443 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-acb2ac77-ab79-4797-8253-d3affaf96243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:28.456154 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4f9db-6094-4c68-bf7d-1408d9556797 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:28.631716 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 849/1692] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:28.633894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb2ac77-ab79-4797-8253-d3affaf96243 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:28.654884 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa74ad-bc9b-4f60-a4e5-9484d9470561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:28.655802 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa74ad-bc9b-4f60-a4e5-9484d9470561 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:28.656381 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa74ad-bc9b-4f60-a4e5-9484d9470561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:28.657441 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa74ad-bc9b-4f60-a4e5-9484d9470561 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:28.658573 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa74ad-bc9b-4f60-a4e5-9484d9470561 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:28.662662 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa74ad-bc9b-4f60-a4e5-9484d9470561 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:28.780529 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 844/1693] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:05:28 2022] POST /identity/v3/users => generated 288 bytes in 547 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:28.790600 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa74ad-bc9b-4f60-a4e5-9484d9470561 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:28.813923 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb2ac77-ab79-4797-8253-d3affaf96243 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:28.829392 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa74ad-bc9b-4f60-a4e5-9484d9470561 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:28.830534 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba31035-f159-4ff6-a113-31c99938947a tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58154460e86d4f4f9b11af2beb3f828a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b9a776450c4463ba8648c8f1a33fdb3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:28.832756 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba31035-f159-4ff6-a113-31c99938947a tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:28.833052 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba31035-f159-4ff6-a113-31c99938947a tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:28.834596 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba31035-f159-4ff6-a113-31c99938947a tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:28.838610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba31035-f159-4ff6-a113-31c99938947a tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:05:28.839825 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba31035-f159-4ff6-a113-31c99938947a tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:28.847058 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba31035-f159-4ff6-a113-31c99938947a tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:28.849973 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948899f-6a39-497d-b53b-ab1acea616a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:28.850616 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948899f-6a39-497d-b53b-ab1acea616a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:28.850896 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948899f-6a39-497d-b53b-ab1acea616a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:28.856012 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948899f-6a39-497d-b53b-ab1acea616a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:28.857893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3948899f-6a39-497d-b53b-ab1acea616a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:28.862387 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3948899f-6a39-497d-b53b-ab1acea616a1 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:28.884528 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 850/1694] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:05:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:05:28.895457 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba31035-f159-4ff6-a113-31c99938947a tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:28.921328 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba31035-f159-4ff6-a113-31c99938947a tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:28.926259 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df003ea2-9eea-4502-82ee-e9069bcb0e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:28.926942 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df003ea2-9eea-4502-82ee-e9069bcb0e99 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:28.927849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df003ea2-9eea-4502-82ee-e9069bcb0e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:28.927849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df003ea2-9eea-4502-82ee-e9069bcb0e99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:28.931823 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df003ea2-9eea-4502-82ee-e9069bcb0e99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:28.933726 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df003ea2-9eea-4502-82ee-e9069bcb0e99 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:28.955765 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 851/1695] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:28.955765 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df003ea2-9eea-4502-82ee-e9069bcb0e99 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:29.005941 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 845/1696] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:05:28 2022] POST /identity/v3/projects => generated 367 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:29.008863 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948899f-6a39-497d-b53b-ab1acea616a1 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:29.022893 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df003ea2-9eea-4502-82ee-e9069bcb0e99 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:29.051516 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:29.180631 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c5619-e163-4749-a71f-62700bcde0e0 tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58154460e86d4f4f9b11af2beb3f828a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b9a776450c4463ba8648c8f1a33fdb3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:29.181189 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c5619-e163-4749-a71f-62700bcde0e0 tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:29.181473 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c5619-e163-4749-a71f-62700bcde0e0 tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:29.181748 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c5619-e163-4749-a71f-62700bcde0e0 tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:29.182404 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c5619-e163-4749-a71f-62700bcde0e0 tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:29.184376 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c5619-e163-4749-a71f-62700bcde0e0 tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:29.224876 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948899f-6a39-497d-b53b-ab1acea616a1 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:29.243186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e04d89-ea2c-4796-9ac4-4eee7a29dfec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:29.244188 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e04d89-ea2c-4796-9ac4-4eee7a29dfec admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:29.244579 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e04d89-ea2c-4796-9ac4-4eee7a29dfec admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:29.244862 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e04d89-ea2c-4796-9ac4-4eee7a29dfec admin admin] PATH_INFO: `/v3/projects/34efddddd5c34103a19ecf2c30945ee2/users/0ad76575b10f43769a0a0950d5a6322f/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:29.274602 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:29.291040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e04d89-ea2c-4796-9ac4-4eee7a29dfec admin admin] RBAC: Authorizing `identity:create_grant(project_id=34efddddd5c34103a19ecf2c30945ee2, user_id=0ad76575b10f43769a0a0950d5a6322f, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:29.295973 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e04d89-ea2c-4796-9ac4-4eee7a29dfec admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:29.343032 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 852/1697] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:05:28 2022] POST /identity/v3/projects => generated 355 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:29.347590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c5619-e163-4749-a71f-62700bcde0e0 tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:29.386068 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 846/1698] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:29 2022] PUT /identity/v3/projects/34efddddd5c34103a19ecf2c30945ee2/users/0ad76575b10f43769a0a0950d5a6322f/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:29.390448 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e04d89-ea2c-4796-9ac4-4eee7a29dfec admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:29.519687 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c5619-e163-4749-a71f-62700bcde0e0 tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:29.556649 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e04d89-ea2c-4796-9ac4-4eee7a29dfec admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:29.568857 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12994323-9f89-459c-8f01-b5a1f962521f tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58154460e86d4f4f9b11af2beb3f828a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b9a776450c4463ba8648c8f1a33fdb3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:29.569491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12994323-9f89-459c-8f01-b5a1f962521f tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:29.569836 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12994323-9f89-459c-8f01-b5a1f962521f tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:29.570061 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12994323-9f89-459c-8f01-b5a1f962521f tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:29.571782 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12994323-9f89-459c-8f01-b5a1f962521f tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:05:29.573428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12994323-9f89-459c-8f01-b5a1f962521f tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:29.577376 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12994323-9f89-459c-8f01-b5a1f962521f tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:29.614528 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 847/1699] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:05:29.615599 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12994323-9f89-459c-8f01-b5a1f962521f tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:29.657537 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b57af0-80e9-4090-92de-042252b7f19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:29.658108 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b57af0-80e9-4090-92de-042252b7f19d admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:29.658360 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b57af0-80e9-4090-92de-042252b7f19d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:29.658635 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b57af0-80e9-4090-92de-042252b7f19d admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:29.659280 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b57af0-80e9-4090-92de-042252b7f19d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:29.676514 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b57af0-80e9-4090-92de-042252b7f19d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:29.798011 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 853/1700] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:05:29 2022] POST /identity/v3/users => generated 290 bytes in 455 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:29.799196 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b57af0-80e9-4090-92de-042252b7f19d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:29.818715 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:29.837626 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b57af0-80e9-4090-92de-042252b7f19d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:29.852739 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12994323-9f89-459c-8f01-b5a1f962521f tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:29.856911 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334631a4-9829-4250-bf84-7bc6d5fe7cd1 tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58154460e86d4f4f9b11af2beb3f828a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b9a776450c4463ba8648c8f1a33fdb3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:29.857811 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334631a4-9829-4250-bf84-7bc6d5fe7cd1 tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:29.858677 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334631a4-9829-4250-bf84-7bc6d5fe7cd1 tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:29.859617 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334631a4-9829-4250-bf84-7bc6d5fe7cd1 tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] PATH_INFO: `/v3/projects/b091f97cd6d44b29ae976f6949914440` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:29.866665 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334631a4-9829-4250-bf84-7bc6d5fe7cd1 tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b091f97cd6d44b29ae976f6949914440)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:29.871115 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f06eb-c0ab-4022-b8e1-49497111e343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:29.874689 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334631a4-9829-4250-bf84-7bc6d5fe7cd1 tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:29.875065 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f06eb-c0ab-4022-b8e1-49497111e343 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:29.875065 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f06eb-c0ab-4022-b8e1-49497111e343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:29.875065 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f06eb-c0ab-4022-b8e1-49497111e343 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:29.875065 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f06eb-c0ab-4022-b8e1-49497111e343 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:29.885975 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f06eb-c0ab-4022-b8e1-49497111e343 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:29.910657 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 848/1701] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:29.911918 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f06eb-c0ab-4022-b8e1-49497111e343 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:29.943581 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f06eb-c0ab-4022-b8e1-49497111e343 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:29.965720 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819eb495-ef55-4f4f-aaa4-7c748c69fa5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:29.966463 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819eb495-ef55-4f4f-aaa4-7c748c69fa5c admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:29.966821 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819eb495-ef55-4f4f-aaa4-7c748c69fa5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:29.967170 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819eb495-ef55-4f4f-aaa4-7c748c69fa5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:29.967903 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819eb495-ef55-4f4f-aaa4-7c748c69fa5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:29.969580 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819eb495-ef55-4f4f-aaa4-7c748c69fa5c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:29.989864 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 849/1702] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:29.991230 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819eb495-ef55-4f4f-aaa4-7c748c69fa5c admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:30.033359 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819eb495-ef55-4f4f-aaa4-7c748c69fa5c admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:30.039624 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c747dbdc-feda-4867-904f-9e6a51075345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:30.040526 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747dbdc-feda-4867-904f-9e6a51075345 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:30.040908 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747dbdc-feda-4867-904f-9e6a51075345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:30.041278 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747dbdc-feda-4867-904f-9e6a51075345 admin admin] PATH_INFO: `/v3/projects/34efddddd5c34103a19ecf2c30945ee2/users/0ad76575b10f43769a0a0950d5a6322f/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:30.043546 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c747dbdc-feda-4867-904f-9e6a51075345 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34efddddd5c34103a19ecf2c30945ee2, user_id=0ad76575b10f43769a0a0950d5a6322f, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:30.046256 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c747dbdc-feda-4867-904f-9e6a51075345 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:30.068096 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-334631a4-9829-4250-bf84-7bc6d5fe7cd1 tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b091f97cd6d44b29ae976f6949914440', 'request_id': 'req-334631a4-9829-4250-bf84-7bc6d5fe7cd1'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:30.077452 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 850/1703] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:29 2022] PUT /identity/v3/projects/34efddddd5c34103a19ecf2c30945ee2/users/0ad76575b10f43769a0a0950d5a6322f/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:30.077990 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-334631a4-9829-4250-bf84-7bc6d5fe7cd1 tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b091f97cd6d44b29ae976f6949914440', 'request_id': 'req-334631a4-9829-4250-bf84-7bc6d5fe7cd1'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:30.079208 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c747dbdc-feda-4867-904f-9e6a51075345 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:30.154899 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-334631a4-9829-4250-bf84-7bc6d5fe7cd1 tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b091f97cd6d44b29ae976f6949914440', 'request_id': 'req-334631a4-9829-4250-bf84-7bc6d5fe7cd1'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:30.201961 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-334631a4-9829-4250-bf84-7bc6d5fe7cd1 tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] The token cache is being invalidate because project b091f97cd6d44b29ae976f6949914440 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:05:30.203830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-334631a4-9829-4250-bf84-7bc6d5fe7cd1 tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:30.207138 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-334631a4-9829-4250-bf84-7bc6d5fe7cd1 tempest-VolumeQuotaClassesTest-224847564 tempest-VolumeQuotaClassesTest-224847564-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:30.212479 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 854/1704] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:05:29 2022] DELETE /identity/v3/projects/b091f97cd6d44b29ae976f6949914440 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:30.215908 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:30.231433 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb467d9-34cc-466c-ad47-929629b604f0 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:30.231433 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb467d9-34cc-466c-ad47-929629b604f0 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:30.231433 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb467d9-34cc-466c-ad47-929629b604f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:30.326715 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-4cb467d9-34cc-466c-ad47-929629b604f0 None None] MFA Rules not processed for user `0ad76575b10f43769a0a0950d5a6322f`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:30.344341 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4cb467d9-34cc-466c-ad47-929629b604f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:30.345231 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c747dbdc-feda-4867-904f-9e6a51075345 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:30.365358 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de9f3fc-f762-4c2b-b911-90bdf63ebf11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:30.367103 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de9f3fc-f762-4c2b-b911-90bdf63ebf11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:30.368507 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de9f3fc-f762-4c2b-b911-90bdf63ebf11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:30.369830 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de9f3fc-f762-4c2b-b911-90bdf63ebf11 admin admin] PATH_INFO: `/v3/projects/0e823339b9834984899b7935b5e7b47d/users/662566981276460595f8f4242a83846b/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:30.432743 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de9f3fc-f762-4c2b-b911-90bdf63ebf11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e823339b9834984899b7935b5e7b47d, user_id=662566981276460595f8f4242a83846b, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:30.432743 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de9f3fc-f762-4c2b-b911-90bdf63ebf11 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:30.500976 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 855/1705] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:30 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:30.501979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb467d9-34cc-466c-ad47-929629b604f0 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:30.503971 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4cb467d9-34cc-466c-ad47-929629b604f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:30.665848 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb467d9-34cc-466c-ad47-929629b604f0 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:30.671616 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824cfdad-474a-4822-b85c-32fe511d4450 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4f6de2065d4fbea3fa141da760f708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:30.672353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cfdad-474a-4822-b85c-32fe511d4450 service placement] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:30.672636 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cfdad-474a-4822-b85c-32fe511d4450 service placement] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:30.672936 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cfdad-474a-4822-b85c-32fe511d4450 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:30.677770 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-824cfdad-474a-4822-b85c-32fe511d4450 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:30.700855 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824cfdad-474a-4822-b85c-32fe511d4450 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:30.704682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824cfdad-474a-4822-b85c-32fe511d4450 service placement] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:30.720423 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 856/1706] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:30.722994 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb83dd0-8543-45a0-b83b-47990df76724 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:30.723521 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb83dd0-8543-45a0-b83b-47990df76724 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:30.724217 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb83dd0-8543-45a0-b83b-47990df76724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:30.748227 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-8cb83dd0-8543-45a0-b83b-47990df76724 None None] MFA Rules not processed for user `c764d9778ef74b709cedbccabac37af0`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:30.749532 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8cb83dd0-8543-45a0-b83b-47990df76724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:30.775597 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 851/1707] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:30 2022] PUT /identity/v3/projects/0e823339b9834984899b7935b5e7b47d/users/662566981276460595f8f4242a83846b/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:30.776905 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de9f3fc-f762-4c2b-b911-90bdf63ebf11 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:30.779904 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3de9f3fc-f762-4c2b-b911-90bdf63ebf11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:30.896446 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 857/1708] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:30 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:30.897161 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb83dd0-8543-45a0-b83b-47990df76724 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:30.908146 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8cb83dd0-8543-45a0-b83b-47990df76724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:30.915118 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de9f3fc-f762-4c2b-b911-90bdf63ebf11 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:30.932274 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6370a74-d1cb-439e-889b-4ee922b08b55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:30.932274 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6370a74-d1cb-439e-889b-4ee922b08b55 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:30.932457 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6370a74-d1cb-439e-889b-4ee922b08b55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:30.932700 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6370a74-d1cb-439e-889b-4ee922b08b55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:30.957110 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb83dd0-8543-45a0-b83b-47990df76724 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:31.198265 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a59f4-11e7-426a-a0a8-4b687be04d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:31.198265 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a59f4-11e7-426a-a0a8-4b687be04d68 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:31.198265 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a59f4-11e7-426a-a0a8-4b687be04d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:31.198265 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a59f4-11e7-426a-a0a8-4b687be04d68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:31.200404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6370a74-d1cb-439e-889b-4ee922b08b55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:31.203011 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6370a74-d1cb-439e-889b-4ee922b08b55 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:31.205048 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a59f4-11e7-426a-a0a8-4b687be04d68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:31.214136 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a59f4-11e7-426a-a0a8-4b687be04d68 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:31.244697 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 852/1709] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:30 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:31.255343 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6370a74-d1cb-439e-889b-4ee922b08b55 service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:31.264054 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 858/1710] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:31.290820 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6370a74-d1cb-439e-889b-4ee922b08b55 service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:31.324299 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452eb64-b53a-4866-b517-98f9945a7a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:31.326163 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452eb64-b53a-4866-b517-98f9945a7a84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:31.327058 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452eb64-b53a-4866-b517-98f9945a7a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:31.328431 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452eb64-b53a-4866-b517-98f9945a7a84 admin admin] PATH_INFO: `/v3/projects/0e823339b9834984899b7935b5e7b47d/users/662566981276460595f8f4242a83846b/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:31.334373 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8452eb64-b53a-4866-b517-98f9945a7a84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e823339b9834984899b7935b5e7b47d, user_id=662566981276460595f8f4242a83846b, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:31.354526 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8452eb64-b53a-4866-b517-98f9945a7a84 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:31.367513 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:31.406083 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 853/1711] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:31 2022] PUT /identity/v3/projects/0e823339b9834984899b7935b5e7b47d/users/662566981276460595f8f4242a83846b/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:31.407702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4a808-fc54-4617-a560-ef2f87817e9a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:31.407992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4a808-fc54-4617-a560-ef2f87817e9a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:31.408294 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4a808-fc54-4617-a560-ef2f87817e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:31.522301 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-81b4a808-fc54-4617-a560-ef2f87817e9a None None] MFA Rules not processed for user `662566981276460595f8f4242a83846b`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:31.524360 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-81b4a808-fc54-4617-a560-ef2f87817e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:31.617788 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452eb64-b53a-4866-b517-98f9945a7a84 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:31.619911 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8452eb64-b53a-4866-b517-98f9945a7a84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:31.786869 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 859/1712] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:31 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:31.798592 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452eb64-b53a-4866-b517-98f9945a7a84 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:31.817068 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4a808-fc54-4617-a560-ef2f87817e9a None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:31.832520 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56cbb1e-763c-4f84-a194-d4d66fee3f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:31.836607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56cbb1e-763c-4f84-a194-d4d66fee3f64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:31.836607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56cbb1e-763c-4f84-a194-d4d66fee3f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:31.836607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56cbb1e-763c-4f84-a194-d4d66fee3f64 admin admin] PATH_INFO: `/v3/users/226b35df09724aaca8c906df71f3bc60` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:31.844725 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56cbb1e-763c-4f84-a194-d4d66fee3f64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=226b35df09724aaca8c906df71f3bc60)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:31.852781 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56cbb1e-763c-4f84-a194-d4d66fee3f64 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:31.938605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:31.985906 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4a808-fc54-4617-a560-ef2f87817e9a None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:31.993061 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4023228c-5e25-457f-8c89-031e7c5362dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:31.993061 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023228c-5e25-457f-8c89-031e7c5362dc service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:31.993061 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023228c-5e25-457f-8c89-031e7c5362dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:31.993061 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023228c-5e25-457f-8c89-031e7c5362dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:32.005089 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4023228c-5e25-457f-8c89-031e7c5362dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:32.006769 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4023228c-5e25-457f-8c89-031e7c5362dc service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:32.026490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 860/1713] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:31 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:05:32.238830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4023228c-5e25-457f-8c89-031e7c5362dc service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:32.250779 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4023228c-5e25-457f-8c89-031e7c5362dc service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:32.269079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a512c-b7a1-4462-b5cb-6397fd3f26eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:32.273488 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a512c-b7a1-4462-b5cb-6397fd3f26eb service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:32.274242 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a512c-b7a1-4462-b5cb-6397fd3f26eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:32.274803 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a512c-b7a1-4462-b5cb-6397fd3f26eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:32.281886 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e56cbb1e-763c-4f84-a194-d4d66fee3f64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '226b35df09724aaca8c906df71f3bc60'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:32.283397 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e56cbb1e-763c-4f84-a194-d4d66fee3f64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '226b35df09724aaca8c906df71f3bc60'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:32.284436 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-037a512c-b7a1-4462-b5cb-6397fd3f26eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:32.318076 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e56cbb1e-763c-4f84-a194-d4d66fee3f64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '226b35df09724aaca8c906df71f3bc60'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:32.362855 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e56cbb1e-763c-4f84-a194-d4d66fee3f64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '226b35df09724aaca8c906df71f3bc60'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:32.370383 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e56cbb1e-763c-4f84-a194-d4d66fee3f64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '226b35df09724aaca8c906df71f3bc60'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:32.459371 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 854/1714] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:05:31 2022] DELETE /identity/v3/users/226b35df09724aaca8c906df71f3bc60 => generated 0 bytes in 842 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:32.463865 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db64a1-0ca7-4804-b484-220742643a5a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:32.467981 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db64a1-0ca7-4804-b484-220742643a5a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:32.469297 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db64a1-0ca7-4804-b484-220742643a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:32.525580 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-a3db64a1-0ca7-4804-b484-220742643a5a None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:32.526743 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a3db64a1-0ca7-4804-b484-220742643a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:32.557245 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a512c-b7a1-4462-b5cb-6397fd3f26eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:32.558908 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a512c-b7a1-4462-b5cb-6397fd3f26eb service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:32.610979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 861/1715] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:05:32.796118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:32.800641 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 855/1716] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:05:32.842101 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a512c-b7a1-4462-b5cb-6397fd3f26eb service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:32.843797 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-037a512c-b7a1-4462-b5cb-6397fd3f26eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:33.097634 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a512c-b7a1-4462-b5cb-6397fd3f26eb service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:33.190179 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c9f54-001d-4c80-9220-06433a862242 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:33.191054 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c9f54-001d-4c80-9220-06433a862242 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:33.191402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c9f54-001d-4c80-9220-06433a862242 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:33.191944 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c9f54-001d-4c80-9220-06433a862242 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:33.238447 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c9f54-001d-4c80-9220-06433a862242 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:33.240289 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c9f54-001d-4c80-9220-06433a862242 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:33.267483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db64a1-0ca7-4804-b484-220742643a5a None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:33.270682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 862/1717] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:33.272764 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a3db64a1-0ca7-4804-b484-220742643a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:33.313594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:33.323505 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db64a1-0ca7-4804-b484-220742643a5a None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:33.336797 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9b6532-4f77-4056-88d8-1dd2cd63c048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:33.337899 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b6532-4f77-4056-88d8-1dd2cd63c048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:33.338340 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b6532-4f77-4056-88d8-1dd2cd63c048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:33.338767 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b6532-4f77-4056-88d8-1dd2cd63c048 admin admin] PATH_INFO: `/v3/users/33776df0eed64572b88df64e500207d7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:33.342085 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9b6532-4f77-4056-88d8-1dd2cd63c048 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33776df0eed64572b88df64e500207d7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:33.344443 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9b6532-4f77-4056-88d8-1dd2cd63c048 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:33.413023 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138b411-ec5e-4e4b-9dad-ffed7c535eca None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:33.413972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138b411-ec5e-4e4b-9dad-ffed7c535eca None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:33.414332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138b411-ec5e-4e4b-9dad-ffed7c535eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:33.474059 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5138b411-ec5e-4e4b-9dad-ffed7c535eca None None] MFA Rules not processed for user `710e49db188845bbae5a183c6657e07f`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:33.477037 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5138b411-ec5e-4e4b-9dad-ffed7c535eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:33.604334 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 863/1718] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:05:33.639300 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138b411-ec5e-4e4b-9dad-ffed7c535eca None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:33.643984 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5138b411-ec5e-4e4b-9dad-ffed7c535eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:33.659097 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-4f9b6532-4f77-4056-88d8-1dd2cd63c048 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33776df0eed64572b88df64e500207d7'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:33.660499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-4f9b6532-4f77-4056-88d8-1dd2cd63c048 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33776df0eed64572b88df64e500207d7'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:33.692390 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-4f9b6532-4f77-4056-88d8-1dd2cd63c048 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33776df0eed64572b88df64e500207d7'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:33.739675 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-4f9b6532-4f77-4056-88d8-1dd2cd63c048 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33776df0eed64572b88df64e500207d7'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:33.739675 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-4f9b6532-4f77-4056-88d8-1dd2cd63c048 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33776df0eed64572b88df64e500207d7'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:33.831669 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 856/1719] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:05:33 2022] DELETE /identity/v3/users/33776df0eed64572b88df64e500207d7 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:33.836843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde2361-e6c6-4a5e-b333-00b732d0c774 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:33.837130 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde2361-e6c6-4a5e-b333-00b732d0c774 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:33.837448 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde2361-e6c6-4a5e-b333-00b732d0c774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:33.893691 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-2dde2361-e6c6-4a5e-b333-00b732d0c774 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:33.896763 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2dde2361-e6c6-4a5e-b333-00b732d0c774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:33.925011 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138b411-ec5e-4e4b-9dad-ffed7c535eca None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:34.137877 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 857/1720] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:05:34.171835 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dde2361-e6c6-4a5e-b333-00b732d0c774 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:34.173626 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2dde2361-e6c6-4a5e-b333-00b732d0c774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:34.187276 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72493449-6a3f-430b-9185-038ebd19d7b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:34.187919 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72493449-6a3f-430b-9185-038ebd19d7b9 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:34.190829 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72493449-6a3f-430b-9185-038ebd19d7b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:34.190829 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72493449-6a3f-430b-9185-038ebd19d7b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:34.198875 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-72493449-6a3f-430b-9185-038ebd19d7b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:34.336341 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dde2361-e6c6-4a5e-b333-00b732d0c774 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:34.352023 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a3f73-63c5-4c06-a18c-9397bc2aae64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:34.352023 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a3f73-63c5-4c06-a18c-9397bc2aae64 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:34.352023 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a3f73-63c5-4c06-a18c-9397bc2aae64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:34.359946 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a3f73-63c5-4c06-a18c-9397bc2aae64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:34.363963 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72493449-6a3f-430b-9185-038ebd19d7b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:34.366304 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72493449-6a3f-430b-9185-038ebd19d7b9 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:34.388853 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0a3f73-63c5-4c06-a18c-9397bc2aae64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:34.390602 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0a3f73-63c5-4c06-a18c-9397bc2aae64 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:34.391573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 864/1721] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:33 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 755 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:34.403630 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 858/1722] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:34.678478 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d3857-567d-471e-b030-f60f6086392d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:34.680984 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d3857-567d-471e-b030-f60f6086392d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:34.681429 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d3857-567d-471e-b030-f60f6086392d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:34.732631 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-556d3857-567d-471e-b030-f60f6086392d None None] MFA Rules not processed for user `83e9308215e94c5e9e615be1f2a113b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:34.736205 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-556d3857-567d-471e-b030-f60f6086392d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:34.802706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a3f73-63c5-4c06-a18c-9397bc2aae64 service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:34.805731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ff0a3f73-63c5-4c06-a18c-9397bc2aae64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:34.827473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a3f73-63c5-4c06-a18c-9397bc2aae64 service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:34.848601 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a1372-b831-46d5-8f5e-add9272cb7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:34.853349 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a1372-b831-46d5-8f5e-add9272cb7c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:34.853349 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a1372-b831-46d5-8f5e-add9272cb7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:34.853349 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a1372-b831-46d5-8f5e-add9272cb7c5 admin admin] PATH_INFO: `/v3/users/2faaa45412584cb2b4adf0e37d77eafd` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:34.853349 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83a1372-b831-46d5-8f5e-add9272cb7c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2faaa45412584cb2b4adf0e37d77eafd)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:34.855062 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83a1372-b831-46d5-8f5e-add9272cb7c5 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:34.957053 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 865/1723] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:34.966911 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16afd57-0d46-4ed6-9dc6-547e8a103a91 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:34.967207 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16afd57-0d46-4ed6-9dc6-547e8a103a91 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:34.967487 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16afd57-0d46-4ed6-9dc6-547e8a103a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:34.987969 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:35.022397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a16afd57-0d46-4ed6-9dc6-547e8a103a91 None None] MFA Rules not processed for user `83e9308215e94c5e9e615be1f2a113b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:35.028521 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a16afd57-0d46-4ed6-9dc6-547e8a103a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:35.036216 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 866/1724] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:05:35.050995 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16afd57-0d46-4ed6-9dc6-547e8a103a91 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:35.060461 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a16afd57-0d46-4ed6-9dc6-547e8a103a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:35.119046 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16afd57-0d46-4ed6-9dc6-547e8a103a91 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:35.136806 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a83a1372-b831-46d5-8f5e-add9272cb7c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2faaa45412584cb2b4adf0e37d77eafd'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:35.143245 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a83a1372-b831-46d5-8f5e-add9272cb7c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2faaa45412584cb2b4adf0e37d77eafd'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:35.143397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d62aaba-c259-4f5b-8b43-6b802de9c5bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:35.143507 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62aaba-c259-4f5b-8b43-6b802de9c5bf service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:35.143765 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62aaba-c259-4f5b-8b43-6b802de9c5bf service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:35.144097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62aaba-c259-4f5b-8b43-6b802de9c5bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:35.153260 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a83a1372-b831-46d5-8f5e-add9272cb7c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2faaa45412584cb2b4adf0e37d77eafd'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:35.158400 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d62aaba-c259-4f5b-8b43-6b802de9c5bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:35.161356 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d62aaba-c259-4f5b-8b43-6b802de9c5bf service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:35.194492 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 867/1725] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:35.198138 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a83a1372-b831-46d5-8f5e-add9272cb7c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2faaa45412584cb2b4adf0e37d77eafd'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:35.201513 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a83a1372-b831-46d5-8f5e-add9272cb7c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2faaa45412584cb2b4adf0e37d77eafd'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:35.230468 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d62aaba-c259-4f5b-8b43-6b802de9c5bf service nova] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:35.230468 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6d62aaba-c259-4f5b-8b43-6b802de9c5bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:35.255544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:35.260590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d62aaba-c259-4f5b-8b43-6b802de9c5bf service nova] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:35.275008 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eafb3f-6adb-447d-a6e5-a0d2699b7654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:35.279030 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eafb3f-6adb-447d-a6e5-a0d2699b7654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:35.279030 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eafb3f-6adb-447d-a6e5-a0d2699b7654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:35.279030 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eafb3f-6adb-447d-a6e5-a0d2699b7654 admin admin] PATH_INFO: `/v3/projects/5354a22bd97e424f8acb938816236ff3` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:35.279030 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eafb3f-6adb-447d-a6e5-a0d2699b7654 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5354a22bd97e424f8acb938816236ff3)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:35.287007 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eafb3f-6adb-447d-a6e5-a0d2699b7654 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:35.307331 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 859/1726] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:05:34 2022] DELETE /identity/v3/users/2faaa45412584cb2b4adf0e37d77eafd => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:05:35.309111 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff821b6a-7cc2-469d-890a-804164d74922 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:35.310027 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff821b6a-7cc2-469d-890a-804164d74922 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:35.311759 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff821b6a-7cc2-469d-890a-804164d74922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:35.361501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-ff821b6a-7cc2-469d-890a-804164d74922 None None] MFA Rules not processed for user `e3ef96f764244e5893f16a74d4169edb`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:35.363751 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ff821b6a-7cc2-469d-890a-804164d74922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:35.473163 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c0eafb3f-6adb-447d-a6e5-a0d2699b7654 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5354a22bd97e424f8acb938816236ff3', 'request_id': 'req-c0eafb3f-6adb-447d-a6e5-a0d2699b7654'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:35.490461 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c0eafb3f-6adb-447d-a6e5-a0d2699b7654 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5354a22bd97e424f8acb938816236ff3', 'request_id': 'req-c0eafb3f-6adb-447d-a6e5-a0d2699b7654'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:35.568711 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c0eafb3f-6adb-447d-a6e5-a0d2699b7654 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5354a22bd97e424f8acb938816236ff3', 'request_id': 'req-c0eafb3f-6adb-447d-a6e5-a0d2699b7654'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:35.618638 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c0eafb3f-6adb-447d-a6e5-a0d2699b7654 admin admin] The token cache is being invalidate because project 5354a22bd97e424f8acb938816236ff3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:05:35.618940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c0eafb3f-6adb-447d-a6e5-a0d2699b7654 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:35.621205 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c0eafb3f-6adb-447d-a6e5-a0d2699b7654 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:35.623368 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 860/1727] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:35.625084 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cdec0-937c-4797-801d-3097d9225185 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:35.625564 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cdec0-937c-4797-801d-3097d9225185 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:35.626117 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cdec0-937c-4797-801d-3097d9225185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:35.626645 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 868/1728] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:05:35 2022] DELETE /identity/v3/projects/5354a22bd97e424f8acb938816236ff3 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:35.631216 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eafb3f-6adb-447d-a6e5-a0d2699b7654 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:35.634025 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c0eafb3f-6adb-447d-a6e5-a0d2699b7654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:35.664389 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-a62cdec0-937c-4797-801d-3097d9225185 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:35.666641 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a62cdec0-937c-4797-801d-3097d9225185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:35.812437 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 861/1729] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:35.816567 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62cdec0-937c-4797-801d-3097d9225185 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:35.822588 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a62cdec0-937c-4797-801d-3097d9225185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:35.834455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eafb3f-6adb-447d-a6e5-a0d2699b7654 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:35.851626 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac2aae-3dee-4484-830c-a9834e8b6a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:35.855104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac2aae-3dee-4484-830c-a9834e8b6a23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:35.856699 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac2aae-3dee-4484-830c-a9834e8b6a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:35.856989 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac2aae-3dee-4484-830c-a9834e8b6a23 admin admin] PATH_INFO: `/v3/users/58154460e86d4f4f9b11af2beb3f828a` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:35.863018 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dac2aae-3dee-4484-830c-a9834e8b6a23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58154460e86d4f4f9b11af2beb3f828a)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:35.869779 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dac2aae-3dee-4484-830c-a9834e8b6a23 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:35.939946 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:36.068087 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62cdec0-937c-4797-801d-3097d9225185 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:36.089737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4b7c7-2b9e-468f-abce-ad444cefb15f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:36.091987 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4b7c7-2b9e-468f-abce-ad444cefb15f service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:36.092435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4b7c7-2b9e-468f-abce-ad444cefb15f service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:36.092575 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4b7c7-2b9e-468f-abce-ad444cefb15f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:36.098872 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-67b4b7c7-2b9e-468f-abce-ad444cefb15f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:36.174998 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-3dac2aae-3dee-4484-830c-a9834e8b6a23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58154460e86d4f4f9b11af2beb3f828a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:36.177629 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-3dac2aae-3dee-4484-830c-a9834e8b6a23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58154460e86d4f4f9b11af2beb3f828a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:36.180828 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-3dac2aae-3dee-4484-830c-a9834e8b6a23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58154460e86d4f4f9b11af2beb3f828a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:36.216101 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-3dac2aae-3dee-4484-830c-a9834e8b6a23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58154460e86d4f4f9b11af2beb3f828a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:36.259155 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-3dac2aae-3dee-4484-830c-a9834e8b6a23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58154460e86d4f4f9b11af2beb3f828a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:36.336921 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b4b7c7-2b9e-468f-abce-ad444cefb15f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:36.338556 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b4b7c7-2b9e-468f-abce-ad444cefb15f service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:36.384237 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 862/1730] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:36.385137 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4b7c7-2b9e-468f-abce-ad444cefb15f service nova] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:36.389542 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-67b4b7c7-2b9e-468f-abce-ad444cefb15f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:36.410473 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 869/1731] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:05:35 2022] DELETE /identity/v3/users/58154460e86d4f4f9b11af2beb3f828a => generated 0 bytes in 777 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:36.410473 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac2aae-3dee-4484-830c-a9834e8b6a23 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:36.410473 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3dac2aae-3dee-4484-830c-a9834e8b6a23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:36.422702 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4b7c7-2b9e-468f-abce-ad444cefb15f service nova] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:36.571404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba60e40-d205-4e9c-91ef-e0dee3530e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:36.573250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba60e40-d205-4e9c-91ef-e0dee3530e1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:36.574451 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba60e40-d205-4e9c-91ef-e0dee3530e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:36.576291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba60e40-d205-4e9c-91ef-e0dee3530e1a admin admin] PATH_INFO: `/v3/projects/ad067e2527ff4e7d9bed04c66c91c1df` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:36.577993 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba60e40-d205-4e9c-91ef-e0dee3530e1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad067e2527ff4e7d9bed04c66c91c1df)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:36.582006 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba60e40-d205-4e9c-91ef-e0dee3530e1a admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:36.632415 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac2aae-3dee-4484-830c-a9834e8b6a23 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:36.645557 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a348fe15-6955-4155-b0ef-081048f9a30d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:36.648547 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348fe15-6955-4155-b0ef-081048f9a30d service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:36.649004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348fe15-6955-4155-b0ef-081048f9a30d service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:36.649824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348fe15-6955-4155-b0ef-081048f9a30d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:36.652748 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a348fe15-6955-4155-b0ef-081048f9a30d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:36.669343 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a348fe15-6955-4155-b0ef-081048f9a30d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:36.684564 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a348fe15-6955-4155-b0ef-081048f9a30d service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:36.776366 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 870/1732] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:36.790690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:36.854264 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0ba60e40-d205-4e9c-91ef-e0dee3530e1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad067e2527ff4e7d9bed04c66c91c1df', 'request_id': 'req-0ba60e40-d205-4e9c-91ef-e0dee3530e1a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:36.943186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0ba60e40-d205-4e9c-91ef-e0dee3530e1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad067e2527ff4e7d9bed04c66c91c1df', 'request_id': 'req-0ba60e40-d205-4e9c-91ef-e0dee3530e1a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:36.966590 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:36.970570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0ba60e40-d205-4e9c-91ef-e0dee3530e1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad067e2527ff4e7d9bed04c66c91c1df', 'request_id': 'req-0ba60e40-d205-4e9c-91ef-e0dee3530e1a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:37.029874 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0ba60e40-d205-4e9c-91ef-e0dee3530e1a admin admin] The token cache is being invalidate because project ad067e2527ff4e7d9bed04c66c91c1df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:05:37.030294 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0ba60e40-d205-4e9c-91ef-e0dee3530e1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:37.034118 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0ba60e40-d205-4e9c-91ef-e0dee3530e1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:37.038205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 863/1733] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:05:36 2022] DELETE /identity/v3/projects/ad067e2527ff4e7d9bed04c66c91c1df => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:37.042742 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a348fe15-6955-4155-b0ef-081048f9a30d service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:37.044153 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a348fe15-6955-4155-b0ef-081048f9a30d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:37.069148 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a348fe15-6955-4155-b0ef-081048f9a30d service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:37.121386 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328370f7-6a1a-4a46-b2a9-311bdc00c97e None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:37.121817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328370f7-6a1a-4a46-b2a9-311bdc00c97e None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:37.122673 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328370f7-6a1a-4a46-b2a9-311bdc00c97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:37.262727 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-328370f7-6a1a-4a46-b2a9-311bdc00c97e None None] MFA Rules not processed for user `8c21ea7d3cbb494886e60a4bd4f39145`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:37.265055 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-328370f7-6a1a-4a46-b2a9-311bdc00c97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:37.309002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c1ae1b-062d-4c60-b722-d2aa91a152b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:37.310330 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1ae1b-062d-4c60-b722-d2aa91a152b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:37.310858 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1ae1b-062d-4c60-b722-d2aa91a152b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:37.312275 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1ae1b-062d-4c60-b722-d2aa91a152b4 admin admin] PATH_INFO: `/v3/users/0ad76575b10f43769a0a0950d5a6322f` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:37.313839 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c1ae1b-062d-4c60-b722-d2aa91a152b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ad76575b10f43769a0a0950d5a6322f)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:37.316334 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c1ae1b-062d-4c60-b722-d2aa91a152b4 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:37.389231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:37.470607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 864/1734] 10.222.0.48 () {58 vars in 1009 bytes} [Fri Sep 9 21:05:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:05:37.555679 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e9c1ae1b-062d-4c60-b722-d2aa91a152b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ad76575b10f43769a0a0950d5a6322f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:37.557298 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e9c1ae1b-062d-4c60-b722-d2aa91a152b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ad76575b10f43769a0a0950d5a6322f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:37.560684 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e9c1ae1b-062d-4c60-b722-d2aa91a152b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ad76575b10f43769a0a0950d5a6322f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:37.581746 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e9c1ae1b-062d-4c60-b722-d2aa91a152b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ad76575b10f43769a0a0950d5a6322f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:37.618070 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e9c1ae1b-062d-4c60-b722-d2aa91a152b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ad76575b10f43769a0a0950d5a6322f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:37.759272 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 871/1735] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:05:37 2022] DELETE /identity/v3/users/0ad76575b10f43769a0a0950d5a6322f => generated 0 bytes in 714 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:37.942910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328370f7-6a1a-4a46-b2a9-311bdc00c97e None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:37.951618 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-328370f7-6a1a-4a46-b2a9-311bdc00c97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:38.087717 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:38.304597 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328370f7-6a1a-4a46-b2a9-311bdc00c97e None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:38.430939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efdbf7b-c251-4614-9762-8238ce2f9f8f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d28fa5d49d44120bd5eaec2d4e5fdbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:38.432010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdbf7b-c251-4614-9762-8238ce2f9f8f service swift] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:38.432656 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdbf7b-c251-4614-9762-8238ce2f9f8f service swift] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:38.433253 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdbf7b-c251-4614-9762-8238ce2f9f8f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:38.435620 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2efdbf7b-c251-4614-9762-8238ce2f9f8f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:38.544329 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efdbf7b-c251-4614-9762-8238ce2f9f8f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:38.548743 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efdbf7b-c251-4614-9762-8238ce2f9f8f service swift] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:38.576614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 865/1736] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Sep 9 21:05:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 634 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 21:05:38.688214 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c1ae1b-062d-4c60-b722-d2aa91a152b4 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:38.693844 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e9c1ae1b-062d-4c60-b722-d2aa91a152b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:38.893108 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c1ae1b-062d-4c60-b722-d2aa91a152b4 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:38.900586 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e59a9-b098-4043-b115-4371826b875f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:38.902125 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e59a9-b098-4043-b115-4371826b875f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:38.902560 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e59a9-b098-4043-b115-4371826b875f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:38.902986 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e59a9-b098-4043-b115-4371826b875f admin admin] PATH_INFO: `/v3/projects/3b9a776450c4463ba8648c8f1a33fdb3` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:38.905641 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e59a9-b098-4043-b115-4371826b875f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b9a776450c4463ba8648c8f1a33fdb3)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:38.908849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e59a9-b098-4043-b115-4371826b875f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:39.078112 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c60e59a9-b098-4043-b115-4371826b875f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b9a776450c4463ba8648c8f1a33fdb3', 'request_id': 'req-c60e59a9-b098-4043-b115-4371826b875f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:39.101161 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c60e59a9-b098-4043-b115-4371826b875f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b9a776450c4463ba8648c8f1a33fdb3', 'request_id': 'req-c60e59a9-b098-4043-b115-4371826b875f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:39.122694 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efdbf7b-c251-4614-9762-8238ce2f9f8f service swift] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:39.125031 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2efdbf7b-c251-4614-9762-8238ce2f9f8f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:39.149276 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:39.183544 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efdbf7b-c251-4614-9762-8238ce2f9f8f service swift] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:39.201266 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c60e59a9-b098-4043-b115-4371826b875f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b9a776450c4463ba8648c8f1a33fdb3', 'request_id': 'req-c60e59a9-b098-4043-b115-4371826b875f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:39.247738 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c60e59a9-b098-4043-b115-4371826b875f admin admin] The token cache is being invalidate because project 3b9a776450c4463ba8648c8f1a33fdb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:05:39.255529 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c60e59a9-b098-4043-b115-4371826b875f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:39.257187 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c60e59a9-b098-4043-b115-4371826b875f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:39.267101 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 872/1737] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:05:38 2022] DELETE /identity/v3/projects/3b9a776450c4463ba8648c8f1a33fdb3 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:39.290763 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5783d-4f36-4682-aeeb-709df9626932 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:39.292097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5783d-4f36-4682-aeeb-709df9626932 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:39.294542 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5783d-4f36-4682-aeeb-709df9626932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:39.368524 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-3da5783d-4f36-4682-aeeb-709df9626932 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:39.375475 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3da5783d-4f36-4682-aeeb-709df9626932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:39.547800 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a67a10-461b-4157-940c-93d80fb994c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:39.548502 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a67a10-461b-4157-940c-93d80fb994c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:39.548782 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a67a10-461b-4157-940c-93d80fb994c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:39.557536 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a67a10-461b-4157-940c-93d80fb994c6 admin admin] PATH_INFO: `/v3/projects/34efddddd5c34103a19ecf2c30945ee2` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:39.562911 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a67a10-461b-4157-940c-93d80fb994c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34efddddd5c34103a19ecf2c30945ee2)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:39.570867 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a67a10-461b-4157-940c-93d80fb994c6 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:39.656910 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 873/1738] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:39.670021 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314370a-8a0a-455d-8adf-6c9c0a1f970e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:39.670837 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314370a-8a0a-455d-8adf-6c9c0a1f970e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:39.673246 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314370a-8a0a-455d-8adf-6c9c0a1f970e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:39.785511 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c314370a-8a0a-455d-8adf-6c9c0a1f970e None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:39.793573 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c314370a-8a0a-455d-8adf-6c9c0a1f970e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:39.843394 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 874/1739] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:39.852420 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314370a-8a0a-455d-8adf-6c9c0a1f970e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:39.895402 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-80a67a10-461b-4157-940c-93d80fb994c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34efddddd5c34103a19ecf2c30945ee2', 'request_id': 'req-80a67a10-461b-4157-940c-93d80fb994c6'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:39.913101 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314370a-8a0a-455d-8adf-6c9c0a1f970e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:40.013050 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-80a67a10-461b-4157-940c-93d80fb994c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34efddddd5c34103a19ecf2c30945ee2', 'request_id': 'req-80a67a10-461b-4157-940c-93d80fb994c6'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:40.021733 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-80a67a10-461b-4157-940c-93d80fb994c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34efddddd5c34103a19ecf2c30945ee2', 'request_id': 'req-80a67a10-461b-4157-940c-93d80fb994c6'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:40.081180 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-80a67a10-461b-4157-940c-93d80fb994c6 admin admin] The token cache is being invalidate because project 34efddddd5c34103a19ecf2c30945ee2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:05:40.082401 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-80a67a10-461b-4157-940c-93d80fb994c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:40.088484 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-80a67a10-461b-4157-940c-93d80fb994c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:40.103122 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 866/1740] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:05:39 2022] DELETE /identity/v3/projects/34efddddd5c34103a19ecf2c30945ee2 => generated 0 bytes in 977 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:40.103951 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4597e4-1f12-4c25-8c18-ea995071927d None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:40.111267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4597e4-1f12-4c25-8c18-ea995071927d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:40.111646 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4597e4-1f12-4c25-8c18-ea995071927d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:40.144866 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a052e-ea63-4946-a243-77a70b79d325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:40.145466 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a052e-ea63-4946-a243-77a70b79d325 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:40.145734 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a052e-ea63-4946-a243-77a70b79d325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:40.146008 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a052e-ea63-4946-a243-77a70b79d325 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:40.146669 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600a052e-ea63-4946-a243-77a70b79d325 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:05:40.153495 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600a052e-ea63-4946-a243-77a70b79d325 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:40.174516 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600a052e-ea63-4946-a243-77a70b79d325 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:40.229048 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-ab4597e4-1f12-4c25-8c18-ea995071927d None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:40.230297 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ab4597e4-1f12-4c25-8c18-ea995071927d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:40.247822 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 875/1741] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:05:40.249060 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a052e-ea63-4946-a243-77a70b79d325 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:40.251528 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-600a052e-ea63-4946-a243-77a70b79d325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:40.266744 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 867/1742] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:40.274967 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3f3ec-779f-4614-bc9d-578e8c6e4bdf None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:40.275247 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3f3ec-779f-4614-bc9d-578e8c6e4bdf None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:40.275507 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3f3ec-779f-4614-bc9d-578e8c6e4bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:40.318063 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a052e-ea63-4946-a243-77a70b79d325 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:40.331904 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-7bd3f3ec-779f-4614-bc9d-578e8c6e4bdf None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:40.333820 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-7bd3f3ec-779f-4614-bc9d-578e8c6e4bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:40.334361 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a58e40-e17d-4245-ad51-196f4da8de55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:40.335639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a58e40-e17d-4245-ad51-196f4da8de55 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:40.335922 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a58e40-e17d-4245-ad51-196f4da8de55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:40.336288 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a58e40-e17d-4245-ad51-196f4da8de55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:40.336925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a58e40-e17d-4245-ad51-196f4da8de55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:40.341408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a58e40-e17d-4245-ad51-196f4da8de55 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:40.349774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 868/1743] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:40.354208 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3f3ec-779f-4614-bc9d-578e8c6e4bdf None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:40.376240 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3f3ec-779f-4614-bc9d-578e8c6e4bdf None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:40.393797 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b339509b-bbd0-4a4d-a580-926c113d568f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:40.394537 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339509b-bbd0-4a4d-a580-926c113d568f admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:40.395023 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339509b-bbd0-4a4d-a580-926c113d568f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:40.395460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339509b-bbd0-4a4d-a580-926c113d568f admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:40.396613 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b339509b-bbd0-4a4d-a580-926c113d568f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:05:40.397127 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b339509b-bbd0-4a4d-a580-926c113d568f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:40.402805 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b339509b-bbd0-4a4d-a580-926c113d568f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:40.428165 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 869/1744] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:05:40.429560 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b339509b-bbd0-4a4d-a580-926c113d568f admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:40.432429 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b339509b-bbd0-4a4d-a580-926c113d568f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:40.501923 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:40.509515 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 876/1745] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:05:40 2022] POST /identity/v3/projects => generated 393 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:40.510294 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a58e40-e17d-4245-ad51-196f4da8de55 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:40.523292 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b339509b-bbd0-4a4d-a580-926c113d568f admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:40.683362 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a58e40-e17d-4245-ad51-196f4da8de55 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:40.688204 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4bcc26-2715-4127-b42f-2f7530beef23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:40.689731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4bcc26-2715-4127-b42f-2f7530beef23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:40.690034 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4bcc26-2715-4127-b42f-2f7530beef23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:40.690304 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4bcc26-2715-4127-b42f-2f7530beef23 admin admin] PATH_INFO: `/v3/projects/c061b12a075e4876a2a455cc048442ad` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:40.695006 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4bcc26-2715-4127-b42f-2f7530beef23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c061b12a075e4876a2a455cc048442ad)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:40.697134 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4bcc26-2715-4127-b42f-2f7530beef23 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:40.709182 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2005ba9-997a-47ba-942c-1f5389951102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:40.709637 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2005ba9-997a-47ba-942c-1f5389951102 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:40.709921 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2005ba9-997a-47ba-942c-1f5389951102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:40.710815 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2005ba9-997a-47ba-942c-1f5389951102 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:40.711611 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2005ba9-997a-47ba-942c-1f5389951102 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:40.713624 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2005ba9-997a-47ba-942c-1f5389951102 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:40.994772 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 877/1746] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:05:40 2022] POST /identity/v3/projects => generated 409 bytes in 485 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:40.995769 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2005ba9-997a-47ba-942c-1f5389951102 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:41.131772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7e4bcc26-2715-4127-b42f-2f7530beef23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c061b12a075e4876a2a455cc048442ad', 'request_id': 'req-7e4bcc26-2715-4127-b42f-2f7530beef23'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:41.181366 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2005ba9-997a-47ba-942c-1f5389951102 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:41.217590 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7e4bcc26-2715-4127-b42f-2f7530beef23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c061b12a075e4876a2a455cc048442ad', 'request_id': 'req-7e4bcc26-2715-4127-b42f-2f7530beef23'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:41.228361 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7e4bcc26-2715-4127-b42f-2f7530beef23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c061b12a075e4876a2a455cc048442ad', 'request_id': 'req-7e4bcc26-2715-4127-b42f-2f7530beef23'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:41.300846 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7e4bcc26-2715-4127-b42f-2f7530beef23 admin admin] The token cache is being invalidate because project c061b12a075e4876a2a455cc048442ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:05:41.301382 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7e4bcc26-2715-4127-b42f-2f7530beef23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:41.308649 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7e4bcc26-2715-4127-b42f-2f7530beef23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:41.332339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 870/1747] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:05:40 2022] DELETE /identity/v3/projects/c061b12a075e4876a2a455cc048442ad => generated 0 bytes in 903 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:41.333674 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4bcc26-2715-4127-b42f-2f7530beef23 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:41.335685 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-7e4bcc26-2715-4127-b42f-2f7530beef23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:41.395271 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637580c-a2b1-41ca-8e12-90c2d833bae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:41.399303 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637580c-a2b1-41ca-8e12-90c2d833bae9 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:41.399303 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637580c-a2b1-41ca-8e12-90c2d833bae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:41.399303 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637580c-a2b1-41ca-8e12-90c2d833bae9 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:41.399303 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4637580c-a2b1-41ca-8e12-90c2d833bae9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:41.403355 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4637580c-a2b1-41ca-8e12-90c2d833bae9 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:41.573754 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 878/1748] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:05:40 2022] POST /identity/v3/users => generated 297 bytes in 578 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:41.575026 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637580c-a2b1-41ca-8e12-90c2d833bae9 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:41.586864 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4637580c-a2b1-41ca-8e12-90c2d833bae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:41.600385 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4bcc26-2715-4127-b42f-2f7530beef23 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:41.616141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e93823-3245-4ffe-bee6-e3b856c33d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:41.618942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e93823-3245-4ffe-bee6-e3b856c33d4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:41.619456 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e93823-3245-4ffe-bee6-e3b856c33d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:41.619965 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e93823-3245-4ffe-bee6-e3b856c33d4e admin admin] PATH_INFO: `/v3/users/d3cb3cdb7c56482a86df55b0d85b4d0f` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:41.625305 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e93823-3245-4ffe-bee6-e3b856c33d4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3cb3cdb7c56482a86df55b0d85b4d0f)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:41.627738 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e93823-3245-4ffe-bee6-e3b856c33d4e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:41.646466 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637580c-a2b1-41ca-8e12-90c2d833bae9 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:41.661300 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8663f0a-969d-47c7-901c-328899f3d8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:41.663997 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8663f0a-969d-47c7-901c-328899f3d8e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:41.665781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8663f0a-969d-47c7-901c-328899f3d8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:41.666159 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8663f0a-969d-47c7-901c-328899f3d8e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:41.667487 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8663f0a-969d-47c7-901c-328899f3d8e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:41.673681 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8663f0a-969d-47c7-901c-328899f3d8e1 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:41.869001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 879/1749] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:05:41 2022] POST /identity/v3/users => generated 305 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:41.892214 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3e747-d33c-4f69-ba18-384d8d11dc61 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:41.899132 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3e747-d33c-4f69-ba18-384d8d11dc61 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:41.902399 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3e747-d33c-4f69-ba18-384d8d11dc61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:41.920458 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b8e93823-3245-4ffe-bee6-e3b856c33d4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3cb3cdb7c56482a86df55b0d85b4d0f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:41.922281 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b8e93823-3245-4ffe-bee6-e3b856c33d4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3cb3cdb7c56482a86df55b0d85b4d0f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:41.943442 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b8e93823-3245-4ffe-bee6-e3b856c33d4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3cb3cdb7c56482a86df55b0d85b4d0f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:41.950784 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-72e3e747-d33c-4f69-ba18-384d8d11dc61 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:41.953020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-72e3e747-d33c-4f69-ba18-384d8d11dc61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:41.975529 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b8e93823-3245-4ffe-bee6-e3b856c33d4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3cb3cdb7c56482a86df55b0d85b4d0f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:41.975529 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b8e93823-3245-4ffe-bee6-e3b856c33d4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3cb3cdb7c56482a86df55b0d85b4d0f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:41.987444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 880/1750] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:41.989968 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f49ca-504a-4af6-8838-26dea11f2776 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:41.990856 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f49ca-504a-4af6-8838-26dea11f2776 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:41.991468 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f49ca-504a-4af6-8838-26dea11f2776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:42.049163 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ab9f49ca-504a-4af6-8838-26dea11f2776 None None] MFA Rules not processed for user `8f5dfac06ac9492a86503ba133c638a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:42.060804 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ab9f49ca-504a-4af6-8838-26dea11f2776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:42.091787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 871/1751] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:05:41 2022] DELETE /identity/v3/users/d3cb3cdb7c56482a86df55b0d85b4d0f => generated 0 bytes in 756 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:42.093211 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e93823-3245-4ffe-bee6-e3b856c33d4e admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:42.094845 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b8e93823-3245-4ffe-bee6-e3b856c33d4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:42.185097 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:42.269661 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 881/1752] 10.222.0.44 () {58 vars in 1015 bytes} [Fri Sep 9 21:05:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:42.301586 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f49ca-504a-4af6-8838-26dea11f2776 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:42.308998 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ab9f49ca-504a-4af6-8838-26dea11f2776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:42.313929 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e93823-3245-4ffe-bee6-e3b856c33d4e admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:42.321685 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2481277-b055-4bbb-9c4b-dbab6d276b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:42.326432 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2481277-b055-4bbb-9c4b-dbab6d276b05 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:42.326432 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2481277-b055-4bbb-9c4b-dbab6d276b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:42.326432 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2481277-b055-4bbb-9c4b-dbab6d276b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:42.326432 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2481277-b055-4bbb-9c4b-dbab6d276b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:42.328270 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2481277-b055-4bbb-9c4b-dbab6d276b05 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:42.360288 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 872/1753] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:42.376571 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45dd5bd-0e00-41e3-8070-e0fc30cbef82 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:42.377889 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45dd5bd-0e00-41e3-8070-e0fc30cbef82 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:42.381553 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45dd5bd-0e00-41e3-8070-e0fc30cbef82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:42.387142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f49ca-504a-4af6-8838-26dea11f2776 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:42.402403 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751482c5-458d-4afc-8a9c-5538ed706fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:42.403740 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751482c5-458d-4afc-8a9c-5538ed706fa3 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:42.404254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751482c5-458d-4afc-8a9c-5538ed706fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:42.405427 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751482c5-458d-4afc-8a9c-5538ed706fa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:42.408425 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751482c5-458d-4afc-8a9c-5538ed706fa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:42.418034 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751482c5-458d-4afc-8a9c-5538ed706fa3 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:42.440586 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-f45dd5bd-0e00-41e3-8070-e0fc30cbef82 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:42.445467 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f45dd5bd-0e00-41e3-8070-e0fc30cbef82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:42.447218 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 882/1754] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:42.451203 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498aef2a-1974-4efc-87de-44eca1428308 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:42.451846 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498aef2a-1974-4efc-87de-44eca1428308 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:42.453200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498aef2a-1974-4efc-87de-44eca1428308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:42.477075 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 873/1755] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:42.480218 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45dd5bd-0e00-41e3-8070-e0fc30cbef82 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:42.513682 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45dd5bd-0e00-41e3-8070-e0fc30cbef82 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:42.514844 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-498aef2a-1974-4efc-87de-44eca1428308 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:42.520117 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-498aef2a-1974-4efc-87de-44eca1428308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:42.531818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0e819-3e95-496c-b9d8-2ac990fc7828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:42.546447 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0e819-3e95-496c-b9d8-2ac990fc7828 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:42.546808 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0e819-3e95-496c-b9d8-2ac990fc7828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:42.547345 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0e819-3e95-496c-b9d8-2ac990fc7828 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:42.548032 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 883/1756] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:42.548925 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea0e819-3e95-496c-b9d8-2ac990fc7828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:42.550597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498aef2a-1974-4efc-87de-44eca1428308 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:42.552749 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea0e819-3e95-496c-b9d8-2ac990fc7828 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:42.555761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-498aef2a-1974-4efc-87de-44eca1428308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:42.593085 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 874/1757] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:42.594489 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0e819-3e95-496c-b9d8-2ac990fc7828 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:42.624831 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0e819-3e95-496c-b9d8-2ac990fc7828 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:42.631293 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d814a8-fb3a-40ad-aedd-5a181aef0532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:42.632922 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d814a8-fb3a-40ad-aedd-5a181aef0532 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:42.643085 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d814a8-fb3a-40ad-aedd-5a181aef0532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:42.643085 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d814a8-fb3a-40ad-aedd-5a181aef0532 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:42.643085 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d814a8-fb3a-40ad-aedd-5a181aef0532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:42.643085 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d814a8-fb3a-40ad-aedd-5a181aef0532 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:42.660106 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 875/1758] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:42.661828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d814a8-fb3a-40ad-aedd-5a181aef0532 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:42.664684 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:42.690323 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d814a8-fb3a-40ad-aedd-5a181aef0532 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:42.698196 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d082ddc-7097-4ee7-b5fb-c135d6dcefaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:42.699747 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d082ddc-7097-4ee7-b5fb-c135d6dcefaf admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:42.699747 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d082ddc-7097-4ee7-b5fb-c135d6dcefaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:42.699921 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d082ddc-7097-4ee7-b5fb-c135d6dcefaf admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:42.701070 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d082ddc-7097-4ee7-b5fb-c135d6dcefaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:05:42.701596 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d082ddc-7097-4ee7-b5fb-c135d6dcefaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:42.704768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d082ddc-7097-4ee7-b5fb-c135d6dcefaf admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:42.739030 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 876/1759] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:05:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:05:42.748623 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d082ddc-7097-4ee7-b5fb-c135d6dcefaf admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:42.765391 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d082ddc-7097-4ee7-b5fb-c135d6dcefaf admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:42.776734 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498aef2a-1974-4efc-87de-44eca1428308 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:42.780795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eaa137-c046-47d2-aa7f-0fbd84e66aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:42.781723 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eaa137-c046-47d2-aa7f-0fbd84e66aec admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:42.782047 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eaa137-c046-47d2-aa7f-0fbd84e66aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:42.782528 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eaa137-c046-47d2-aa7f-0fbd84e66aec admin admin] PATH_INFO: `/v3/projects/70a8e6a050654249b3f33179c225947b/users/364798a6a1e14c44b68f63851927c014/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:42.788612 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609ee5c-be2e-458d-ba32-c8f1c7a94dfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:42.788612 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609ee5c-be2e-458d-ba32-c8f1c7a94dfd service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:42.788612 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609ee5c-be2e-458d-ba32-c8f1c7a94dfd service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:42.793377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609ee5c-be2e-458d-ba32-c8f1c7a94dfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:42.808310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:42.813825 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609ee5c-be2e-458d-ba32-c8f1c7a94dfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:42.822472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609ee5c-be2e-458d-ba32-c8f1c7a94dfd service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:42.833322 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eaa137-c046-47d2-aa7f-0fbd84e66aec admin admin] RBAC: Authorizing `identity:create_grant(project_id=70a8e6a050654249b3f33179c225947b, user_id=364798a6a1e14c44b68f63851927c014, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:42.842202 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eaa137-c046-47d2-aa7f-0fbd84e66aec admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:42.862838 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 884/1760] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:42.864677 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609ee5c-be2e-458d-ba32-c8f1c7a94dfd service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:42.889305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609ee5c-be2e-458d-ba32-c8f1c7a94dfd service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:42.908480 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b15f69-9d43-4d64-ad3f-c9cc515b8569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:42.911423 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b15f69-9d43-4d64-ad3f-c9cc515b8569 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:42.911423 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b15f69-9d43-4d64-ad3f-c9cc515b8569 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:42.912159 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b15f69-9d43-4d64-ad3f-c9cc515b8569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:42.922059 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 877/1761] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:42 2022] PUT /identity/v3/projects/70a8e6a050654249b3f33179c225947b/users/364798a6a1e14c44b68f63851927c014/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:42.923689 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eaa137-c046-47d2-aa7f-0fbd84e66aec admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:42.962343 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b15f69-9d43-4d64-ad3f-c9cc515b8569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:42.970007 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b15f69-9d43-4d64-ad3f-c9cc515b8569 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:43.055570 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 885/1762] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:43.059621 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b15f69-9d43-4d64-ad3f-c9cc515b8569 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:43.105338 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b15f69-9d43-4d64-ad3f-c9cc515b8569 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:43.154095 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eaa137-c046-47d2-aa7f-0fbd84e66aec admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:43.168020 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdaa335-468e-4702-b6cf-0651974330ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:43.169090 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdaa335-468e-4702-b6cf-0651974330ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:43.169713 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdaa335-468e-4702-b6cf-0651974330ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:43.170173 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdaa335-468e-4702-b6cf-0651974330ac admin admin] PATH_INFO: `/v3/projects/fca4e790f8d14e3e8038e402f2ba2959/users/ad32d78ac4f9435a8ef0e96689be950f/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:43.236414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdaa335-468e-4702-b6cf-0651974330ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=fca4e790f8d14e3e8038e402f2ba2959, user_id=ad32d78ac4f9435a8ef0e96689be950f, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:43.241281 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdaa335-468e-4702-b6cf-0651974330ac admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:43.318539 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29cfe0-5bda-480c-87ce-32937963a643 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:43.319182 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29cfe0-5bda-480c-87ce-32937963a643 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:43.321398 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29cfe0-5bda-480c-87ce-32937963a643 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:43.321896 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29cfe0-5bda-480c-87ce-32937963a643 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:43.345842 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b29cfe0-5bda-480c-87ce-32937963a643 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:43.347456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b29cfe0-5bda-480c-87ce-32937963a643 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:43.349879 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 878/1763] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:42 2022] PUT /identity/v3/projects/fca4e790f8d14e3e8038e402f2ba2959/users/ad32d78ac4f9435a8ef0e96689be950f/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:43.364795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdaa335-468e-4702-b6cf-0651974330ac admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:43.372204 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 886/1764] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:43.374365 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a516d62-71b7-4c1c-ad73-faecca9b3ade None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:43.374805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a516d62-71b7-4c1c-ad73-faecca9b3ade None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:43.376329 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a516d62-71b7-4c1c-ad73-faecca9b3ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:43.453436 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-6a516d62-71b7-4c1c-ad73-faecca9b3ade None None] MFA Rules not processed for user `364798a6a1e14c44b68f63851927c014`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:43.456864 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6a516d62-71b7-4c1c-ad73-faecca9b3ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:43.686635 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdaa335-468e-4702-b6cf-0651974330ac admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:43.709817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36c8c5-4f5f-413d-8f71-d78c8df58913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:43.715843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36c8c5-4f5f-413d-8f71-d78c8df58913 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:43.716156 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36c8c5-4f5f-413d-8f71-d78c8df58913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:43.716439 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36c8c5-4f5f-413d-8f71-d78c8df58913 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:43.719457 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f36c8c5-4f5f-413d-8f71-d78c8df58913 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:43.736385 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f36c8c5-4f5f-413d-8f71-d78c8df58913 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:43.747646 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 887/1765] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:43.747646 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88043b92-988f-4878-a152-b340f3904188 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:43.748386 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88043b92-988f-4878-a152-b340f3904188 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:43.752314 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88043b92-988f-4878-a152-b340f3904188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:43.773928 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:43.827590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-88043b92-988f-4878-a152-b340f3904188 None None] MFA Rules not processed for user `ad32d78ac4f9435a8ef0e96689be950f`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:43.829695 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-88043b92-988f-4878-a152-b340f3904188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:43.863922 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 879/1766] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:05:43 2022] POST /identity/v3/projects => generated 385 bytes in 513 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:43.865486 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36c8c5-4f5f-413d-8f71-d78c8df58913 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:43.894542 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:44.000728 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 888/1767] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:43 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:44.002022 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88043b92-988f-4878-a152-b340f3904188 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:44.048286 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88043b92-988f-4878-a152-b340f3904188 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:44.073178 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36c8c5-4f5f-413d-8f71-d78c8df58913 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:44.083881 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b56535-7a50-41a7-b177-4476fa9789f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:44.090348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b56535-7a50-41a7-b177-4476fa9789f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:44.090348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b56535-7a50-41a7-b177-4476fa9789f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:44.090348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b56535-7a50-41a7-b177-4476fa9789f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:44.090348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b56535-7a50-41a7-b177-4476fa9789f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:44.099817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7195330-dfb7-478b-99db-55a015be22e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:44.100496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7195330-dfb7-478b-99db-55a015be22e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:44.101682 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7195330-dfb7-478b-99db-55a015be22e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:44.101682 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7195330-dfb7-478b-99db-55a015be22e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:44.101878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7195330-dfb7-478b-99db-55a015be22e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:44.110643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b56535-7a50-41a7-b177-4476fa9789f4 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:44.112424 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7195330-dfb7-478b-99db-55a015be22e2 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:44.149583 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:44.266710 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 889/1768] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:05:44 2022] POST /identity/v3/users => generated 293 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:44.266710 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b56535-7a50-41a7-b177-4476fa9789f4 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:44.279475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 880/1769] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:05:43 2022] POST /identity/v3/projects => generated 391 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:44.286235 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551b7b8-d9de-4ee2-8d85-71e6cc06f97f None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:44.286235 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551b7b8-d9de-4ee2-8d85-71e6cc06f97f None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:44.286235 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551b7b8-d9de-4ee2-8d85-71e6cc06f97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:44.294707 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b56535-7a50-41a7-b177-4476fa9789f4 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:44.339075 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-d551b7b8-d9de-4ee2-8d85-71e6cc06f97f None None] MFA Rules not processed for user `710e49db188845bbae5a183c6657e07f`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:44.340527 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d551b7b8-d9de-4ee2-8d85-71e6cc06f97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:44.537852 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:44.555129 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2868b0-9fb9-4bc0-9acd-a9e61f434e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:44.555771 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2868b0-9fb9-4bc0-9acd-a9e61f434e46 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:44.556051 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2868b0-9fb9-4bc0-9acd-a9e61f434e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:44.556520 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2868b0-9fb9-4bc0-9acd-a9e61f434e46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:44.565406 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 881/1770] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:44 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:44.566062 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551b7b8-d9de-4ee2-8d85-71e6cc06f97f None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:44.570678 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2868b0-9fb9-4bc0-9acd-a9e61f434e46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:44.593146 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2868b0-9fb9-4bc0-9acd-a9e61f434e46 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:44.618927 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551b7b8-d9de-4ee2-8d85-71e6cc06f97f None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:44.647145 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888884ac-4c2b-43dc-9285-0f839b770dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:44.647948 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888884ac-4c2b-43dc-9285-0f839b770dbf admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:44.648580 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888884ac-4c2b-43dc-9285-0f839b770dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:44.648886 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888884ac-4c2b-43dc-9285-0f839b770dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:44.649644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888884ac-4c2b-43dc-9285-0f839b770dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:44.653657 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888884ac-4c2b-43dc-9285-0f839b770dbf admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:44.680568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 882/1771] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:44.681686 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888884ac-4c2b-43dc-9285-0f839b770dbf admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:44.698870 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888884ac-4c2b-43dc-9285-0f839b770dbf admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:44.704589 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44ee3e-f671-4d49-ad2d-a55933227870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:44.705630 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44ee3e-f671-4d49-ad2d-a55933227870 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:44.706059 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44ee3e-f671-4d49-ad2d-a55933227870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:44.706566 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44ee3e-f671-4d49-ad2d-a55933227870 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:44.718003 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed44ee3e-f671-4d49-ad2d-a55933227870 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:44.720307 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed44ee3e-f671-4d49-ad2d-a55933227870 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:44.747241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 890/1772] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:05:44 2022] POST /identity/v3/projects => generated 409 bytes in 480 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:44.752784 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2868b0-9fb9-4bc0-9acd-a9e61f434e46 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:44.868829 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:44.880852 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 883/1773] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:05:44 2022] POST /identity/v3/users => generated 302 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:44.884302 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44ee3e-f671-4d49-ad2d-a55933227870 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:44.909373 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44ee3e-f671-4d49-ad2d-a55933227870 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:44.937541 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:45.016231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2868b0-9fb9-4bc0-9acd-a9e61f434e46 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:45.030752 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25ef6a-d516-445b-8ce2-30025ea14802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:45.032523 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25ef6a-d516-445b-8ce2-30025ea14802 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:45.043566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25ef6a-d516-445b-8ce2-30025ea14802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:45.043566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25ef6a-d516-445b-8ce2-30025ea14802 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:45.043566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a25ef6a-d516-445b-8ce2-30025ea14802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:45.043566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a25ef6a-d516-445b-8ce2-30025ea14802 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:45.052892 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09bf9c-7346-44cc-9525-429c754128e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:45.053900 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09bf9c-7346-44cc-9525-429c754128e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:45.055170 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09bf9c-7346-44cc-9525-429c754128e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:45.055662 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09bf9c-7346-44cc-9525-429c754128e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:45.056616 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc09bf9c-7346-44cc-9525-429c754128e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:45.060821 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc09bf9c-7346-44cc-9525-429c754128e0 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:45.071025 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 891/1774] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:45.073107 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25ef6a-d516-445b-8ce2-30025ea14802 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:45.096232 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25ef6a-d516-445b-8ce2-30025ea14802 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:45.103321 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87414303-2ba6-472b-a5a4-0eabe48795b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:45.104885 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87414303-2ba6-472b-a5a4-0eabe48795b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:45.104885 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87414303-2ba6-472b-a5a4-0eabe48795b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:45.105435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87414303-2ba6-472b-a5a4-0eabe48795b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:45.106449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87414303-2ba6-472b-a5a4-0eabe48795b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:45.109026 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87414303-2ba6-472b-a5a4-0eabe48795b5 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:45.125835 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 892/1775] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:45.127331 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87414303-2ba6-472b-a5a4-0eabe48795b5 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:45.142176 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87414303-2ba6-472b-a5a4-0eabe48795b5 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:45.156910 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8516582-6472-4bb3-9282-d7717c2fb926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:45.157937 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8516582-6472-4bb3-9282-d7717c2fb926 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:45.158388 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8516582-6472-4bb3-9282-d7717c2fb926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:45.158938 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8516582-6472-4bb3-9282-d7717c2fb926 admin admin] PATH_INFO: `/v3/projects/226b1e5048664e46be6dfdc40804aa05/users/d434dcd5a4c646d9bedae2fff8a1b69a/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:45.184824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8516582-6472-4bb3-9282-d7717c2fb926 admin admin] RBAC: Authorizing `identity:create_grant(project_id=226b1e5048664e46be6dfdc40804aa05, user_id=d434dcd5a4c646d9bedae2fff8a1b69a, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:45.198747 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 884/1776] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:05:44 2022] POST /identity/v3/users => generated 311 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:45.199794 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09bf9c-7346-44cc-9525-429c754128e0 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:45.206156 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8516582-6472-4bb3-9282-d7717c2fb926 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:45.218906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09bf9c-7346-44cc-9525-429c754128e0 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:45.224840 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957e6f4b-9ad8-4341-89d6-d2685888d709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:45.225239 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e6f4b-9ad8-4341-89d6-d2685888d709 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:45.225555 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e6f4b-9ad8-4341-89d6-d2685888d709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:45.225888 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e6f4b-9ad8-4341-89d6-d2685888d709 admin admin] PATH_INFO: `/v3/projects/c63fec25e1184800b7d25bc187b2b063/users/323ef20684234326b4f31fb223fef957/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:45.250786 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957e6f4b-9ad8-4341-89d6-d2685888d709 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c63fec25e1184800b7d25bc187b2b063, user_id=323ef20684234326b4f31fb223fef957, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:45.254017 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957e6f4b-9ad8-4341-89d6-d2685888d709 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:45.320132 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 893/1777] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:45 2022] PUT /identity/v3/projects/226b1e5048664e46be6dfdc40804aa05/users/d434dcd5a4c646d9bedae2fff8a1b69a/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:45.320132 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8516582-6472-4bb3-9282-d7717c2fb926 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:45.368129 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 885/1778] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:45 2022] PUT /identity/v3/projects/c63fec25e1184800b7d25bc187b2b063/users/323ef20684234326b4f31fb223fef957/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:45.370492 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753cc3a-cad3-4fc6-9462-a6d02fa94c7d None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:45.370492 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753cc3a-cad3-4fc6-9462-a6d02fa94c7d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:45.370492 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753cc3a-cad3-4fc6-9462-a6d02fa94c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:45.478818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-0753cc3a-cad3-4fc6-9462-a6d02fa94c7d None None] MFA Rules not processed for user `d434dcd5a4c646d9bedae2fff8a1b69a`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:45.491328 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0753cc3a-cad3-4fc6-9462-a6d02fa94c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:45.514404 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8516582-6472-4bb3-9282-d7717c2fb926 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:45.588500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:45.602011 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:45.620560 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb5536-6e17-4943-a093-d8498619d372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:45.621193 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb5536-6e17-4943-a093-d8498619d372 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:45.621637 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb5536-6e17-4943-a093-d8498619d372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:45.621948 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb5536-6e17-4943-a093-d8498619d372 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:45.622596 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb5536-6e17-4943-a093-d8498619d372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:45.624278 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb5536-6e17-4943-a093-d8498619d372 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:45.661882 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 894/1779] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:45.663031 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb5536-6e17-4943-a093-d8498619d372 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:45.699291 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb5536-6e17-4943-a093-d8498619d372 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:45.720481 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 886/1780] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:45.730514 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753cc3a-cad3-4fc6-9462-a6d02fa94c7d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:45.734512 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da989c-b126-4960-a75e-8780b4047201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:45.735526 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da989c-b126-4960-a75e-8780b4047201 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:45.736576 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da989c-b126-4960-a75e-8780b4047201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:45.737445 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da989c-b126-4960-a75e-8780b4047201 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:45.739389 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66da989c-b126-4960-a75e-8780b4047201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:45.743851 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66da989c-b126-4960-a75e-8780b4047201 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:45.767963 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753cc3a-cad3-4fc6-9462-a6d02fa94c7d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:45.781121 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f6814-2da2-4c0f-a729-fcc2be58cde2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:45.781278 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 895/1781] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:45.781396 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f6814-2da2-4c0f-a729-fcc2be58cde2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:45.781396 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f6814-2da2-4c0f-a729-fcc2be58cde2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:45.781396 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f6814-2da2-4c0f-a729-fcc2be58cde2 admin admin] PATH_INFO: `/v3/projects/0d6aeb92851c44a69579a436efacddda/users/c715ae48efc440258c7014c7c52de669/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:45.783401 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da989c-b126-4960-a75e-8780b4047201 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:45.798468 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da989c-b126-4960-a75e-8780b4047201 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:45.816863 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a635e-65b1-401c-a225-753f65251a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:45.817422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a635e-65b1-401c-a225-753f65251a12 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:45.817673 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a635e-65b1-401c-a225-753f65251a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:45.817943 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a635e-65b1-401c-a225-753f65251a12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:45.818715 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024a635e-65b1-401c-a225-753f65251a12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:45.821346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024a635e-65b1-401c-a225-753f65251a12 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:45.837137 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f6814-2da2-4c0f-a729-fcc2be58cde2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d6aeb92851c44a69579a436efacddda, user_id=c715ae48efc440258c7014c7c52de669, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:45.839937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f6814-2da2-4c0f-a729-fcc2be58cde2 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:45.976597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 896/1782] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:05:45 2022] POST /identity/v3/projects => generated 385 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:45.979575 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a635e-65b1-401c-a225-753f65251a12 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:45.982820 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 887/1783] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:45 2022] PUT /identity/v3/projects/0d6aeb92851c44a69579a436efacddda/users/c715ae48efc440258c7014c7c52de669/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:45.983740 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f6814-2da2-4c0f-a729-fcc2be58cde2 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:45.988715 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-927f6814-2da2-4c0f-a729-fcc2be58cde2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:46.188753 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a635e-65b1-401c-a225-753f65251a12 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:46.242597 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:46.251306 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f6814-2da2-4c0f-a729-fcc2be58cde2 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:46.263058 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f40fdc-ea57-4f5a-a05c-c322133faaae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:46.264050 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f40fdc-ea57-4f5a-a05c-c322133faaae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:46.265703 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f40fdc-ea57-4f5a-a05c-c322133faaae admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:46.266207 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f40fdc-ea57-4f5a-a05c-c322133faaae admin admin] PATH_INFO: `/v3/projects/24f926d61a1f434691d24e5445d60dda` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:46.268653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f40fdc-ea57-4f5a-a05c-c322133faaae admin admin] RBAC: Authorizing `identity:delete_project(project_id=24f926d61a1f434691d24e5445d60dda)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:46.271947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f40fdc-ea57-4f5a-a05c-c322133faaae admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:46.318974 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deae774-edfe-41ff-949d-468c299f462e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:46.320593 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deae774-edfe-41ff-949d-468c299f462e admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:46.321190 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deae774-edfe-41ff-949d-468c299f462e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:46.322010 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deae774-edfe-41ff-949d-468c299f462e admin admin] PATH_INFO: `/v3/projects/c63fec25e1184800b7d25bc187b2b063/users/323ef20684234326b4f31fb223fef957/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:46.325305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deae774-edfe-41ff-949d-468c299f462e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c63fec25e1184800b7d25bc187b2b063, user_id=323ef20684234326b4f31fb223fef957, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:46.329389 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deae774-edfe-41ff-949d-468c299f462e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:46.363059 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 897/1784] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:45 2022] PUT /identity/v3/projects/c63fec25e1184800b7d25bc187b2b063/users/323ef20684234326b4f31fb223fef957/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:46.363059 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deae774-edfe-41ff-949d-468c299f462e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:46.400192 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:46.483299 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deae774-edfe-41ff-949d-468c299f462e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:46.491710 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe34bb5-ddc9-4928-8103-9ea214829b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:46.511930 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe34bb5-ddc9-4928-8103-9ea214829b97 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:46.513412 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe34bb5-ddc9-4928-8103-9ea214829b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:46.513667 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe34bb5-ddc9-4928-8103-9ea214829b97 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:46.514369 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe34bb5-ddc9-4928-8103-9ea214829b97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:46.517412 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe34bb5-ddc9-4928-8103-9ea214829b97 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:46.535653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-d5f40fdc-ea57-4f5a-a05c-c322133faaae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24f926d61a1f434691d24e5445d60dda', 'request_id': 'req-d5f40fdc-ea57-4f5a-a05c-c322133faaae'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:46.647176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-d5f40fdc-ea57-4f5a-a05c-c322133faaae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24f926d61a1f434691d24e5445d60dda', 'request_id': 'req-d5f40fdc-ea57-4f5a-a05c-c322133faaae'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:46.656308 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-d5f40fdc-ea57-4f5a-a05c-c322133faaae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24f926d61a1f434691d24e5445d60dda', 'request_id': 'req-d5f40fdc-ea57-4f5a-a05c-c322133faaae'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:46.667417 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 898/1785] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:05:46 2022] POST /identity/v3/users => generated 299 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:46.671950 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe34bb5-ddc9-4928-8103-9ea214829b97 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:46.685567 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-d5f40fdc-ea57-4f5a-a05c-c322133faaae admin admin] The token cache is being invalidate because project 24f926d61a1f434691d24e5445d60dda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:05:46.686215 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-d5f40fdc-ea57-4f5a-a05c-c322133faaae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:46.687103 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-d5f40fdc-ea57-4f5a-a05c-c322133faaae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:46.690150 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 888/1786] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:05:45 2022] DELETE /identity/v3/projects/24f926d61a1f434691d24e5445d60dda => generated 0 bytes in 707 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:46.693880 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f76cad-3eda-475a-81e3-8b13eb5246ed None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:46.697301 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f76cad-3eda-475a-81e3-8b13eb5246ed None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:46.697301 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f76cad-3eda-475a-81e3-8b13eb5246ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:46.706409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe34bb5-ddc9-4928-8103-9ea214829b97 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:46.766328 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-96f76cad-3eda-475a-81e3-8b13eb5246ed None None] MFA Rules not processed for user `323ef20684234326b4f31fb223fef957`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:46.767814 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-96f76cad-3eda-475a-81e3-8b13eb5246ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:46.928503 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beffdaa-bb22-4b04-827d-c42fb8c92ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:46.930205 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beffdaa-bb22-4b04-827d-c42fb8c92ed8 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:46.933731 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beffdaa-bb22-4b04-827d-c42fb8c92ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:46.934204 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 889/1787] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:46 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:46.935775 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12222eaa-7f73-4b3b-9ad9-18e42b5289db None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:46.936282 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beffdaa-bb22-4b04-827d-c42fb8c92ed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:46.939981 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beffdaa-bb22-4b04-827d-c42fb8c92ed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:46.944136 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beffdaa-bb22-4b04-827d-c42fb8c92ed8 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:46.944413 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:46.944923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12222eaa-7f73-4b3b-9ad9-18e42b5289db None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:46.945231 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12222eaa-7f73-4b3b-9ad9-18e42b5289db None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:46.946696 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 890/1788] 10.222.0.48 () {54 vars in 921 bytes} [Fri Sep 9 21:05:46 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:05:46.947448 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12222eaa-7f73-4b3b-9ad9-18e42b5289db None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:46.950351 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-12222eaa-7f73-4b3b-9ad9-18e42b5289db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:46.967663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 899/1789] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:46.969173 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beffdaa-bb22-4b04-827d-c42fb8c92ed8 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:46.984855 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8beffdaa-bb22-4b04-827d-c42fb8c92ed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:47.008909 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12222eaa-7f73-4b3b-9ad9-18e42b5289db None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:47.022703 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0513b7-9546-4b34-811f-d24dce578c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:47.023592 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0513b7-9546-4b34-811f-d24dce578c03 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:47.024115 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0513b7-9546-4b34-811f-d24dce578c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:47.024937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0513b7-9546-4b34-811f-d24dce578c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:47.026306 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0513b7-9546-4b34-811f-d24dce578c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:47.029923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0513b7-9546-4b34-811f-d24dce578c03 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:47.038974 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beffdaa-bb22-4b04-827d-c42fb8c92ed8 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:47.046495 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9eafcc-fac1-4cc4-a240-1ab584c3690b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:47.046495 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9eafcc-fac1-4cc4-a240-1ab584c3690b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:47.046495 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9eafcc-fac1-4cc4-a240-1ab584c3690b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:47.046495 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9eafcc-fac1-4cc4-a240-1ab584c3690b admin admin] PATH_INFO: `/v3/users/662566981276460595f8f4242a83846b` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:47.065371 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9eafcc-fac1-4cc4-a240-1ab584c3690b admin admin] RBAC: Authorizing `identity:delete_user(user_id=662566981276460595f8f4242a83846b)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:47.071609 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9eafcc-fac1-4cc4-a240-1ab584c3690b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:47.075291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 891/1790] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:47.076655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0513b7-9546-4b34-811f-d24dce578c03 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:47.081580 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8f0513b7-9546-4b34-811f-d24dce578c03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:47.213521 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0513b7-9546-4b34-811f-d24dce578c03 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:47.220116 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25101ef-0313-4905-8dd6-df39d8f1dd66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:47.220116 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25101ef-0313-4905-8dd6-df39d8f1dd66 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:47.220116 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25101ef-0313-4905-8dd6-df39d8f1dd66 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:47.220116 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25101ef-0313-4905-8dd6-df39d8f1dd66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:47.253671 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25101ef-0313-4905-8dd6-df39d8f1dd66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:47.256133 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25101ef-0313-4905-8dd6-df39d8f1dd66 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:47.283977 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 892/1791] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:47 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:47.288191 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25101ef-0313-4905-8dd6-df39d8f1dd66 service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:47.288191 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a25101ef-0313-4905-8dd6-df39d8f1dd66 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:47.330236 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25101ef-0313-4905-8dd6-df39d8f1dd66 service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:47.348366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75351fa2-7ffc-41b7-9f48-8472dda6372a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:47.349050 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75351fa2-7ffc-41b7-9f48-8472dda6372a admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:47.349424 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75351fa2-7ffc-41b7-9f48-8472dda6372a admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:47.349747 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75351fa2-7ffc-41b7-9f48-8472dda6372a admin admin] PATH_INFO: `/v3/projects/0d6aeb92851c44a69579a436efacddda/users/c715ae48efc440258c7014c7c52de669/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:47.370638 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-dd9eafcc-fac1-4cc4-a240-1ab584c3690b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '662566981276460595f8f4242a83846b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:47.371281 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75351fa2-7ffc-41b7-9f48-8472dda6372a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d6aeb92851c44a69579a436efacddda, user_id=c715ae48efc440258c7014c7c52de669, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:47.372578 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-dd9eafcc-fac1-4cc4-a240-1ab584c3690b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '662566981276460595f8f4242a83846b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:47.373262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-dd9eafcc-fac1-4cc4-a240-1ab584c3690b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '662566981276460595f8f4242a83846b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:47.374951 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75351fa2-7ffc-41b7-9f48-8472dda6372a admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:47.391530 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-dd9eafcc-fac1-4cc4-a240-1ab584c3690b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '662566981276460595f8f4242a83846b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:47.423903 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 893/1792] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:47 2022] PUT /identity/v3/projects/0d6aeb92851c44a69579a436efacddda/users/c715ae48efc440258c7014c7c52de669/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:47.425560 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7ae59-1636-4777-a98c-2a0bd6af5bfe None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:47.425853 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7ae59-1636-4777-a98c-2a0bd6af5bfe None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:47.426131 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7ae59-1636-4777-a98c-2a0bd6af5bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:47.426684 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-dd9eafcc-fac1-4cc4-a240-1ab584c3690b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '662566981276460595f8f4242a83846b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:47.470033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-4eb7ae59-1636-4777-a98c-2a0bd6af5bfe None None] MFA Rules not processed for user `613b845a9b4d432daf8d478fc3970708`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:47.471492 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-4eb7ae59-1636-4777-a98c-2a0bd6af5bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:47.540473 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 900/1793] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:05:46 2022] DELETE /identity/v3/users/662566981276460595f8f4242a83846b => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:47.541389 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9eafcc-fac1-4cc4-a240-1ab584c3690b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:47.546475 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-dd9eafcc-fac1-4cc4-a240-1ab584c3690b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:47.624112 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:47.665954 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:47.672772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 894/1794] 10.222.0.48 () {58 vars in 1018 bytes} [Fri Sep 9 21:05:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:47.672772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8c056-b443-4f39-8ef3-0463a80636de None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:47.672772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8c056-b443-4f39-8ef3-0463a80636de None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:47.672772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8c056-b443-4f39-8ef3-0463a80636de None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:47.676157 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 895/1795] 10.222.0.48 () {54 vars in 912 bytes} [Fri Sep 9 21:05:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:05:47.679189 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c136b17-7581-4c52-8f5b-76aa69cdb871 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:47.680868 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c136b17-7581-4c52-8f5b-76aa69cdb871 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:47.682004 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c136b17-7581-4c52-8f5b-76aa69cdb871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:47.709945 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9eafcc-fac1-4cc4-a240-1ab584c3690b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:47.721955 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e7d69-c0ca-4210-81c0-3089dd296935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:47.722644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e7d69-c0ca-4210-81c0-3089dd296935 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:47.722644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e7d69-c0ca-4210-81c0-3089dd296935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:47.723055 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e7d69-c0ca-4210-81c0-3089dd296935 admin admin] PATH_INFO: `/v3/projects/133eb16a74d0449f9c734b0cd4aaf3cc/users/8672fc0906f84bf79d2b41ea2705883c/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:47.745677 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-1c136b17-7581-4c52-8f5b-76aa69cdb871 None None] MFA Rules not processed for user `c715ae48efc440258c7014c7c52de669`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:47.748639 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-1c136b17-7581-4c52-8f5b-76aa69cdb871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:47.759398 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e7d69-c0ca-4210-81c0-3089dd296935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=133eb16a74d0449f9c734b0cd4aaf3cc, user_id=8672fc0906f84bf79d2b41ea2705883c, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:47.761684 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e7d69-c0ca-4210-81c0-3089dd296935 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:47.857671 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 896/1796] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:47 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:47.858796 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c136b17-7581-4c52-8f5b-76aa69cdb871 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:47.862474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-1c136b17-7581-4c52-8f5b-76aa69cdb871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:47.912403 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 901/1797] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:47 2022] PUT /identity/v3/projects/133eb16a74d0449f9c734b0cd4aaf3cc/users/8672fc0906f84bf79d2b41ea2705883c/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:47.914813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20df87d-3ac6-45ab-b95b-e681a65dbd64 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:47.915235 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20df87d-3ac6-45ab-b95b-e681a65dbd64 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:47.915509 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20df87d-3ac6-45ab-b95b-e681a65dbd64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:47.941876 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e20df87d-3ac6-45ab-b95b-e681a65dbd64 None None] MFA Rules not processed for user `8f5dfac06ac9492a86503ba133c638a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:47.943602 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e20df87d-3ac6-45ab-b95b-e681a65dbd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:48.016546 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c136b17-7581-4c52-8f5b-76aa69cdb871 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:48.047981 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 902/1798] 10.222.0.48 () {58 vars in 1009 bytes} [Fri Sep 9 21:05:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:48.049055 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20df87d-3ac6-45ab-b95b-e681a65dbd64 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:48.051083 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e20df87d-3ac6-45ab-b95b-e681a65dbd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:48.192514 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20df87d-3ac6-45ab-b95b-e681a65dbd64 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:48.202381 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cee37e-904f-4392-ab35-812b3284a780 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:48.203097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cee37e-904f-4392-ab35-812b3284a780 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:48.203382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cee37e-904f-4392-ab35-812b3284a780 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:48.203660 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cee37e-904f-4392-ab35-812b3284a780 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:48.209132 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8d7a2-2d5d-46b8-b2ff-01006aab6215 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:48.209132 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8d7a2-2d5d-46b8-b2ff-01006aab6215 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:48.209132 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8d7a2-2d5d-46b8-b2ff-01006aab6215 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:48.209827 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8d7a2-2d5d-46b8-b2ff-01006aab6215 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:48.216534 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cee37e-904f-4392-ab35-812b3284a780 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:48.219102 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cee37e-904f-4392-ab35-812b3284a780 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:48.232543 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-85b8d7a2-2d5d-46b8-b2ff-01006aab6215 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:48.238108 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 903/1799] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:48.238108 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cee37e-904f-4392-ab35-812b3284a780 service nova] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:48.238108 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-66cee37e-904f-4392-ab35-812b3284a780 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:48.248740 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cee37e-904f-4392-ab35-812b3284a780 service nova] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:48.253542 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231cc0c4-c47d-4c7d-8b91-675396836cfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:48.255258 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231cc0c4-c47d-4c7d-8b91-675396836cfe service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:48.255547 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231cc0c4-c47d-4c7d-8b91-675396836cfe service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:48.256577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231cc0c4-c47d-4c7d-8b91-675396836cfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:48.430081 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8d7a2-2d5d-46b8-b2ff-01006aab6215 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:48.431566 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8d7a2-2d5d-46b8-b2ff-01006aab6215 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:48.439439 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231cc0c4-c47d-4c7d-8b91-675396836cfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:48.441187 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231cc0c4-c47d-4c7d-8b91-675396836cfe service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:48.458674 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 897/1800] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:47 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:48.461712 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7238e-e8f9-4bf1-b75a-37c6120548c1 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:48.462670 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7238e-e8f9-4bf1-b75a-37c6120548c1 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:48.463052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7238e-e8f9-4bf1-b75a-37c6120548c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:48.488225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 904/1801] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:48 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:48.488225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231cc0c4-c47d-4c7d-8b91-675396836cfe service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:48.499045 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-84c7238e-e8f9-4bf1-b75a-37c6120548c1 None None] MFA Rules not processed for user `e3ef96f764244e5893f16a74d4169edb`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:48.500599 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-84c7238e-e8f9-4bf1-b75a-37c6120548c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:48.672649 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 898/1802] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:48.676170 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c7238e-e8f9-4bf1-b75a-37c6120548c1 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:48.680353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231cc0c4-c47d-4c7d-8b91-675396836cfe service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:48.687392 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-84c7238e-e8f9-4bf1-b75a-37c6120548c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:48.696459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:48.700229 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d2be5-d3ca-471c-817f-fb503c53e46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:48.700229 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d2be5-d3ca-471c-817f-fb503c53e46e admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:48.700229 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d2be5-d3ca-471c-817f-fb503c53e46e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:48.700843 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d2be5-d3ca-471c-817f-fb503c53e46e admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:48.701198 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947d2be5-d3ca-471c-817f-fb503c53e46e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:48.709149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947d2be5-d3ca-471c-817f-fb503c53e46e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:48.743185 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c7238e-e8f9-4bf1-b75a-37c6120548c1 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:48.744854 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 905/1803] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:48.750307 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d2be5-d3ca-471c-817f-fb503c53e46e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:48.761074 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c82d350-a8de-428b-83d9-307ef1b7ffcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:48.763188 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82d350-a8de-428b-83d9-307ef1b7ffcf service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:48.764479 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82d350-a8de-428b-83d9-307ef1b7ffcf service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:48.764916 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82d350-a8de-428b-83d9-307ef1b7ffcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:48.784884 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d2be5-d3ca-471c-817f-fb503c53e46e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:48.788907 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c82d350-a8de-428b-83d9-307ef1b7ffcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:48.790957 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c82d350-a8de-428b-83d9-307ef1b7ffcf service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:48.798757 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a5a2e-44ee-407e-bfb9-26f6ccc662cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:48.799806 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a5a2e-44ee-407e-bfb9-26f6ccc662cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:48.800275 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a5a2e-44ee-407e-bfb9-26f6ccc662cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:48.801058 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a5a2e-44ee-407e-bfb9-26f6ccc662cb admin admin] PATH_INFO: `/v3/projects/133eb16a74d0449f9c734b0cd4aaf3cc/users/8672fc0906f84bf79d2b41ea2705883c/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:48.805379 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28a5a2e-44ee-407e-bfb9-26f6ccc662cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=133eb16a74d0449f9c734b0cd4aaf3cc, user_id=8672fc0906f84bf79d2b41ea2705883c, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:48.810095 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28a5a2e-44ee-407e-bfb9-26f6ccc662cb admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:48.823583 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 899/1804] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:48.855527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 906/1805] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:48 2022] PUT /identity/v3/projects/133eb16a74d0449f9c734b0cd4aaf3cc/users/8672fc0906f84bf79d2b41ea2705883c/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:48.867158 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df09b39-c93a-41cf-b14d-13883a574aef None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:48.867514 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df09b39-c93a-41cf-b14d-13883a574aef None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:48.867854 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df09b39-c93a-41cf-b14d-13883a574aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:48.911766 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:48.961957 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-3df09b39-c93a-41cf-b14d-13883a574aef None None] MFA Rules not processed for user `8672fc0906f84bf79d2b41ea2705883c`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:48.968690 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3df09b39-c93a-41cf-b14d-13883a574aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:49.170200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a5a2e-44ee-407e-bfb9-26f6ccc662cb admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:49.267731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 900/1806] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:48 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:49.317870 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a5a2e-44ee-407e-bfb9-26f6ccc662cb admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:49.344146 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda58353-2f0b-4adf-9df9-301283feac8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:49.346392 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda58353-2f0b-4adf-9df9-301283feac8b service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:49.348512 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda58353-2f0b-4adf-9df9-301283feac8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:49.349033 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda58353-2f0b-4adf-9df9-301283feac8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:49.370154 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bda58353-2f0b-4adf-9df9-301283feac8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:49.566282 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda58353-2f0b-4adf-9df9-301283feac8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:49.567780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda58353-2f0b-4adf-9df9-301283feac8b service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:49.620148 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 907/1807] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:49 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:49.634712 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df09b39-c93a-41cf-b14d-13883a574aef None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:49.652379 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df09b39-c93a-41cf-b14d-13883a574aef None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:49.661161 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a1b6b0-79e8-4fe5-992b-98d501c9b26b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:49.666160 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1b6b0-79e8-4fe5-992b-98d501c9b26b service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:49.667517 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1b6b0-79e8-4fe5-992b-98d501c9b26b service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:49.667935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1b6b0-79e8-4fe5-992b-98d501c9b26b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:49.692051 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda58353-2f0b-4adf-9df9-301283feac8b service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:49.694667 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bda58353-2f0b-4adf-9df9-301283feac8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:49.711862 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a1b6b0-79e8-4fe5-992b-98d501c9b26b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:49.717293 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a1b6b0-79e8-4fe5-992b-98d501c9b26b service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:49.750929 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:49.758441 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 901/1808] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:49 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:49.846857 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a1b6b0-79e8-4fe5-992b-98d501c9b26b service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:50.016322 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda58353-2f0b-4adf-9df9-301283feac8b service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:50.031350 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:50.040582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22ea98e-536e-420c-a7f4-5b79a54f3607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:50.041154 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ea98e-536e-420c-a7f4-5b79a54f3607 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:50.041496 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ea98e-536e-420c-a7f4-5b79a54f3607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:50.041705 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ea98e-536e-420c-a7f4-5b79a54f3607 admin admin] PATH_INFO: `/v3/projects/0e823339b9834984899b7935b5e7b47d` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:50.043564 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22ea98e-536e-420c-a7f4-5b79a54f3607 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e823339b9834984899b7935b5e7b47d)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:50.064480 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a1b6b0-79e8-4fe5-992b-98d501c9b26b service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:50.072845 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22ea98e-536e-420c-a7f4-5b79a54f3607 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:50.078232 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da874369-d987-4ec0-a646-0d62ffa82e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:50.078986 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da874369-d987-4ec0-a646-0d62ffa82e68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:50.079430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da874369-d987-4ec0-a646-0d62ffa82e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:50.079854 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da874369-d987-4ec0-a646-0d62ffa82e68 admin admin] PATH_INFO: `/v3/users/d434dcd5a4c646d9bedae2fff8a1b69a` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:50.082222 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da874369-d987-4ec0-a646-0d62ffa82e68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d434dcd5a4c646d9bedae2fff8a1b69a)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:50.084631 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da874369-d987-4ec0-a646-0d62ffa82e68 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:50.320123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a22ea98e-536e-420c-a7f4-5b79a54f3607 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e823339b9834984899b7935b5e7b47d', 'request_id': 'req-a22ea98e-536e-420c-a7f4-5b79a54f3607'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:50.320553 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-da874369-d987-4ec0-a646-0d62ffa82e68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd434dcd5a4c646d9bedae2fff8a1b69a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:50.320553 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-da874369-d987-4ec0-a646-0d62ffa82e68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd434dcd5a4c646d9bedae2fff8a1b69a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:50.320886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a22ea98e-536e-420c-a7f4-5b79a54f3607 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e823339b9834984899b7935b5e7b47d', 'request_id': 'req-a22ea98e-536e-420c-a7f4-5b79a54f3607'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:50.327658 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-da874369-d987-4ec0-a646-0d62ffa82e68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd434dcd5a4c646d9bedae2fff8a1b69a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:50.346965 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-da874369-d987-4ec0-a646-0d62ffa82e68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd434dcd5a4c646d9bedae2fff8a1b69a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:50.360949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-da874369-d987-4ec0-a646-0d62ffa82e68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd434dcd5a4c646d9bedae2fff8a1b69a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:50.385997 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a22ea98e-536e-420c-a7f4-5b79a54f3607 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e823339b9834984899b7935b5e7b47d', 'request_id': 'req-a22ea98e-536e-420c-a7f4-5b79a54f3607'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:50.422142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a22ea98e-536e-420c-a7f4-5b79a54f3607 admin admin] The token cache is being invalidate because project 0e823339b9834984899b7935b5e7b47d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:05:50.423343 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a22ea98e-536e-420c-a7f4-5b79a54f3607 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:50.425270 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a22ea98e-536e-420c-a7f4-5b79a54f3607 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:50.437404 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 908/1809] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:05:49 2022] DELETE /identity/v3/projects/0e823339b9834984899b7935b5e7b47d => generated 0 bytes in 743 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:50.441196 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa3d8c-0aa8-4469-ac0c-7cb0c91ff6e3 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:50.441500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa3d8c-0aa8-4469-ac0c-7cb0c91ff6e3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:50.443995 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa3d8c-0aa8-4469-ac0c-7cb0c91ff6e3 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:50.449254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 909/1810] 10.222.0.48 () {54 vars in 921 bytes} [Fri Sep 9 21:05:50 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:05:50.452847 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a727363-2754-46a7-9d3b-9b3052cc3962 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:50.455097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a727363-2754-46a7-9d3b-9b3052cc3962 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:50.455450 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a727363-2754-46a7-9d3b-9b3052cc3962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:50.503059 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5a727363-2754-46a7-9d3b-9b3052cc3962 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:50.507505 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5a727363-2754-46a7-9d3b-9b3052cc3962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:50.521565 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 902/1811] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:05:49 2022] DELETE /identity/v3/users/d434dcd5a4c646d9bedae2fff8a1b69a => generated 0 bytes in 675 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:50.528388 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896bf544-c1c4-4c27-877a-1377446243df None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:50.528954 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896bf544-c1c4-4c27-877a-1377446243df None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:50.530083 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896bf544-c1c4-4c27-877a-1377446243df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:50.590852 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-896bf544-c1c4-4c27-877a-1377446243df None None] MFA Rules not processed for user `613b845a9b4d432daf8d478fc3970708`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:50.593102 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-896bf544-c1c4-4c27-877a-1377446243df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:50.635241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 910/1812] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:50.636667 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf59022-d8a7-4cbb-a06e-db1fe860d6c6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:50.637016 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf59022-d8a7-4cbb-a06e-db1fe860d6c6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:50.637298 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf59022-d8a7-4cbb-a06e-db1fe860d6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:50.720981 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-8bf59022-d8a7-4cbb-a06e-db1fe860d6c6 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:50.725001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8bf59022-d8a7-4cbb-a06e-db1fe860d6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:50.926300 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 903/1813] 10.222.0.48 () {58 vars in 1018 bytes} [Fri Sep 9 21:05:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:50.928002 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c53342-7490-4747-882c-75077fea89ea None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:50.929695 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c53342-7490-4747-882c-75077fea89ea None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:50.930105 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c53342-7490-4747-882c-75077fea89ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:50.969465 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 911/1814] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:51.014786 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf59022-d8a7-4cbb-a06e-db1fe860d6c6 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:51.016960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8bf59022-d8a7-4cbb-a06e-db1fe860d6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:51.020011 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-28c53342-7490-4747-882c-75077fea89ea None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:51.031485 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-28c53342-7490-4747-882c-75077fea89ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:51.076272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 904/1815] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:51.077485 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c53342-7490-4747-882c-75077fea89ea None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:51.114775 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c53342-7490-4747-882c-75077fea89ea None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:51.125539 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:51.142153 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebecf7e3-393a-4d9d-b96c-f691b3aa0c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:51.142153 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebecf7e3-393a-4d9d-b96c-f691b3aa0c7f admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:51.142153 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebecf7e3-393a-4d9d-b96c-f691b3aa0c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:51.142153 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebecf7e3-393a-4d9d-b96c-f691b3aa0c7f admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:51.142433 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebecf7e3-393a-4d9d-b96c-f691b3aa0c7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:05:51.142433 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebecf7e3-393a-4d9d-b96c-f691b3aa0c7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:51.148000 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebecf7e3-393a-4d9d-b96c-f691b3aa0c7f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:51.180628 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 905/1816] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:05:51.182677 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebecf7e3-393a-4d9d-b96c-f691b3aa0c7f admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:51.207167 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebecf7e3-393a-4d9d-b96c-f691b3aa0c7f admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:51.219849 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadaf94d-679e-46a2-85dc-6a9c8baa5b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:51.220717 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadaf94d-679e-46a2-85dc-6a9c8baa5b31 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:51.222337 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadaf94d-679e-46a2-85dc-6a9c8baa5b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:51.224674 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadaf94d-679e-46a2-85dc-6a9c8baa5b31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:51.227270 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadaf94d-679e-46a2-85dc-6a9c8baa5b31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:51.230593 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadaf94d-679e-46a2-85dc-6a9c8baa5b31 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:51.238002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf59022-d8a7-4cbb-a06e-db1fe860d6c6 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:51.244036 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6566ff0-116d-43a7-8f34-7fea1485c1e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:51.244784 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6566ff0-116d-43a7-8f34-7fea1485c1e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:51.245469 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6566ff0-116d-43a7-8f34-7fea1485c1e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:51.245984 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6566ff0-116d-43a7-8f34-7fea1485c1e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:51.296229 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6566ff0-116d-43a7-8f34-7fea1485c1e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:51.322578 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6566ff0-116d-43a7-8f34-7fea1485c1e1 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:51.356550 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 912/1817] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:05:51.383751 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 906/1818] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:05:51 2022] POST /identity/v3/projects => generated 383 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:51.391453 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6566ff0-116d-43a7-8f34-7fea1485c1e1 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:51.415568 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6566ff0-116d-43a7-8f34-7fea1485c1e1 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:51.512468 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc20d0d8-bcfc-4d79-8aca-995595e57b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:51.513455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20d0d8-bcfc-4d79-8aca-995595e57b24 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:51.514158 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20d0d8-bcfc-4d79-8aca-995595e57b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:51.514158 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20d0d8-bcfc-4d79-8aca-995595e57b24 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:51.514938 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc20d0d8-bcfc-4d79-8aca-995595e57b24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:51.517039 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc20d0d8-bcfc-4d79-8aca-995595e57b24 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:51.632285 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadaf94d-679e-46a2-85dc-6a9c8baa5b31 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:51.632579 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 913/1819] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:05:51 2022] POST /identity/v3/users => generated 292 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:05:51.674435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadaf94d-679e-46a2-85dc-6a9c8baa5b31 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:51.697484 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c5ac4-47d1-4e0e-8aad-64ecfb16790e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:51.698177 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c5ac4-47d1-4e0e-8aad-64ecfb16790e admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:51.698526 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c5ac4-47d1-4e0e-8aad-64ecfb16790e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:51.699073 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c5ac4-47d1-4e0e-8aad-64ecfb16790e admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:51.699964 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0c5ac4-47d1-4e0e-8aad-64ecfb16790e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:51.701919 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0c5ac4-47d1-4e0e-8aad-64ecfb16790e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:51.709344 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:51.737488 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 907/1820] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:51.739898 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc20d0d8-bcfc-4d79-8aca-995595e57b24 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:51.753494 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc20d0d8-bcfc-4d79-8aca-995595e57b24 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:51.760636 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd3b71-53af-474a-bcca-f75c096e11c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:51.761569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd3b71-53af-474a-bcca-f75c096e11c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:51.762373 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd3b71-53af-474a-bcca-f75c096e11c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:51.762961 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd3b71-53af-474a-bcca-f75c096e11c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:51.764471 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd3b71-53af-474a-bcca-f75c096e11c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:51.766441 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd3b71-53af-474a-bcca-f75c096e11c2 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:51.780883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 914/1821] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:51.784859 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c5ac4-47d1-4e0e-8aad-64ecfb16790e admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:51.797054 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c5ac4-47d1-4e0e-8aad-64ecfb16790e admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:51.813971 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e209cc8-d608-44fc-aca0-e5b8d3058ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:51.820688 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e209cc8-d608-44fc-aca0-e5b8d3058ee5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:51.821550 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e209cc8-d608-44fc-aca0-e5b8d3058ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:51.821950 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e209cc8-d608-44fc-aca0-e5b8d3058ee5 admin admin] PATH_INFO: `/v3/projects/b54ac4223a9245d690a7841f52c31029/users/35fa3f1d4f7241a0b55281bfa3c31f60/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:51.872716 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e209cc8-d608-44fc-aca0-e5b8d3058ee5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b54ac4223a9245d690a7841f52c31029, user_id=35fa3f1d4f7241a0b55281bfa3c31f60, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:51.878205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e209cc8-d608-44fc-aca0-e5b8d3058ee5 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:52.029930 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 908/1822] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:51 2022] PUT /identity/v3/projects/b54ac4223a9245d690a7841f52c31029/users/35fa3f1d4f7241a0b55281bfa3c31f60/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:52.035390 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83990b34-1426-4b3c-afc9-b03d11134bb6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:52.035842 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83990b34-1426-4b3c-afc9-b03d11134bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:52.036655 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83990b34-1426-4b3c-afc9-b03d11134bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:52.083959 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-83990b34-1426-4b3c-afc9-b03d11134bb6 None None] MFA Rules not processed for user `35fa3f1d4f7241a0b55281bfa3c31f60`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:52.085461 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-83990b34-1426-4b3c-afc9-b03d11134bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:52.226396 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e209cc8-d608-44fc-aca0-e5b8d3058ee5 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:52.230440 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0e209cc8-d608-44fc-aca0-e5b8d3058ee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:52.249188 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 915/1823] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:52.251757 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83990b34-1426-4b3c-afc9-b03d11134bb6 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:52.407396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83990b34-1426-4b3c-afc9-b03d11134bb6 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:52.440228 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a252a-c799-4b63-8392-d9456f059f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:52.440881 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a252a-c799-4b63-8392-d9456f059f9f admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:52.441342 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a252a-c799-4b63-8392-d9456f059f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:52.441690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a252a-c799-4b63-8392-d9456f059f9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:52.442370 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a252a-c799-4b63-8392-d9456f059f9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:52.444438 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a252a-c799-4b63-8392-d9456f059f9f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:52.458083 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e209cc8-d608-44fc-aca0-e5b8d3058ee5 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:52.470925 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c3ea3-681c-496e-a20b-d1e7781577db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:52.470925 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c3ea3-681c-496e-a20b-d1e7781577db service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:52.470925 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c3ea3-681c-496e-a20b-d1e7781577db service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:52.470925 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c3ea3-681c-496e-a20b-d1e7781577db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:52.478887 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c3ea3-681c-496e-a20b-d1e7781577db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:52.480574 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c3ea3-681c-496e-a20b-d1e7781577db service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:52.507893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 909/1824] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:52.601589 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c3ea3-681c-496e-a20b-d1e7781577db service nova] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:52.603578 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-779c3ea3-681c-496e-a20b-d1e7781577db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:52.614854 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 916/1825] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:05:52 2022] POST /identity/v3/projects => generated 385 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:52.615761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a252a-c799-4b63-8392-d9456f059f9f admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:52.636647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c3ea3-681c-496e-a20b-d1e7781577db service nova] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:52.698880 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:52.813634 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cabc559-5ce5-4410-abfd-b699b869829c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:52.813634 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabc559-5ce5-4410-abfd-b699b869829c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:52.813634 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabc559-5ce5-4410-abfd-b699b869829c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:52.813634 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabc559-5ce5-4410-abfd-b699b869829c admin admin] PATH_INFO: `/v3/projects/226b1e5048664e46be6dfdc40804aa05` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:52.814133 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cabc559-5ce5-4410-abfd-b699b869829c admin admin] RBAC: Authorizing `identity:delete_project(project_id=226b1e5048664e46be6dfdc40804aa05)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:52.815087 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cabc559-5ce5-4410-abfd-b699b869829c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:52.816104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a252a-c799-4b63-8392-d9456f059f9f admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:52.824317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e2b97-be64-4822-85c3-877a6f791f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:52.825088 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e2b97-be64-4822-85c3-877a6f791f74 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:52.825463 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e2b97-be64-4822-85c3-877a6f791f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:52.826093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e2b97-be64-4822-85c3-877a6f791f74 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:52.827022 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282e2b97-be64-4822-85c3-877a6f791f74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:52.829490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282e2b97-be64-4822-85c3-877a6f791f74 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:52.932351 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 917/1826] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:05:52 2022] POST /identity/v3/users => generated 293 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:52.937554 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e2b97-be64-4822-85c3-877a6f791f74 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:52.974211 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e2b97-be64-4822-85c3-877a6f791f74 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:52.990644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fe50c-eefe-4207-8522-fb2e3059a0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:52.992441 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fe50c-eefe-4207-8522-fb2e3059a0d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:52.993510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fe50c-eefe-4207-8522-fb2e3059a0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:52.994534 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fe50c-eefe-4207-8522-fb2e3059a0d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:52.995748 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203fe50c-eefe-4207-8522-fb2e3059a0d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:52.998619 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203fe50c-eefe-4207-8522-fb2e3059a0d4 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:53.032447 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 918/1827] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:53.036499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1cabc559-5ce5-4410-abfd-b699b869829c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '226b1e5048664e46be6dfdc40804aa05', 'request_id': 'req-1cabc559-5ce5-4410-abfd-b699b869829c'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:53.037920 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fe50c-eefe-4207-8522-fb2e3059a0d4 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:53.052360 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fe50c-eefe-4207-8522-fb2e3059a0d4 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:53.132312 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1cabc559-5ce5-4410-abfd-b699b869829c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '226b1e5048664e46be6dfdc40804aa05', 'request_id': 'req-1cabc559-5ce5-4410-abfd-b699b869829c'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:53.136785 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:53.145466 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1cabc559-5ce5-4410-abfd-b699b869829c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '226b1e5048664e46be6dfdc40804aa05', 'request_id': 'req-1cabc559-5ce5-4410-abfd-b699b869829c'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:53.196023 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3975c5a-7d4f-45dc-9671-65b408ebdd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:53.196864 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3975c5a-7d4f-45dc-9671-65b408ebdd9c admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:53.197139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3975c5a-7d4f-45dc-9671-65b408ebdd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:53.197419 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3975c5a-7d4f-45dc-9671-65b408ebdd9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:53.198634 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3975c5a-7d4f-45dc-9671-65b408ebdd9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:53.201463 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3975c5a-7d4f-45dc-9671-65b408ebdd9c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:53.222869 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 919/1828] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:53.226789 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3975c5a-7d4f-45dc-9671-65b408ebdd9c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:53.234577 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1cabc559-5ce5-4410-abfd-b699b869829c admin admin] The token cache is being invalidate because project 226b1e5048664e46be6dfdc40804aa05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:05:53.234878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1cabc559-5ce5-4410-abfd-b699b869829c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:53.237454 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1cabc559-5ce5-4410-abfd-b699b869829c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:53.240611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 910/1829] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:05:52 2022] DELETE /identity/v3/projects/226b1e5048664e46be6dfdc40804aa05 => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:53.247841 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3975c5a-7d4f-45dc-9671-65b408ebdd9c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:53.254390 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cabc559-5ce5-4410-abfd-b699b869829c admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:53.254695 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a008cb-4f0e-423a-aac6-a01a07c627a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:53.255545 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a008cb-4f0e-423a-aac6-a01a07c627a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:53.257369 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a008cb-4f0e-423a-aac6-a01a07c627a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:53.258908 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a008cb-4f0e-423a-aac6-a01a07c627a6 admin admin] PATH_INFO: `/v3/projects/80069bd1dee942c7b4af1d06a3efa656/users/d6a927f4c03041b1b2996368f7a3fa80/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:53.260192 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-1cabc559-5ce5-4410-abfd-b699b869829c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:53.282789 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a008cb-4f0e-423a-aac6-a01a07c627a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80069bd1dee942c7b4af1d06a3efa656, user_id=d6a927f4c03041b1b2996368f7a3fa80, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:53.287507 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a008cb-4f0e-423a-aac6-a01a07c627a6 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:53.296640 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cabc559-5ce5-4410-abfd-b699b869829c admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:53.319781 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1079f87a-6bd3-4d45-9bda-26e17aed8a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:53.327713 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079f87a-6bd3-4d45-9bda-26e17aed8a86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:53.328009 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079f87a-6bd3-4d45-9bda-26e17aed8a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:53.328332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079f87a-6bd3-4d45-9bda-26e17aed8a86 admin admin] PATH_INFO: `/v3/users/8672fc0906f84bf79d2b41ea2705883c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:53.353927 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1079f87a-6bd3-4d45-9bda-26e17aed8a86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8672fc0906f84bf79d2b41ea2705883c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:53.356339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1079f87a-6bd3-4d45-9bda-26e17aed8a86 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:53.436363 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 920/1830] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:53 2022] PUT /identity/v3/projects/80069bd1dee942c7b4af1d06a3efa656/users/d6a927f4c03041b1b2996368f7a3fa80/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:53.448122 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20073d-d213-410a-97b1-11e82f3742f4 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:53.448447 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20073d-d213-410a-97b1-11e82f3742f4 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:53.448746 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20073d-d213-410a-97b1-11e82f3742f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:53.530735 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-2b20073d-d213-410a-97b1-11e82f3742f4 None None] MFA Rules not processed for user `d6a927f4c03041b1b2996368f7a3fa80`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:53.537125 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2b20073d-d213-410a-97b1-11e82f3742f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:53.634912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 921/1831] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:05:53.635867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20073d-d213-410a-97b1-11e82f3742f4 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:53.638973 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2b20073d-d213-410a-97b1-11e82f3742f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:53.733507 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20073d-d213-410a-97b1-11e82f3742f4 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:53.739202 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99734b-d164-4ba7-b43d-60191e9957fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:53.739903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99734b-d164-4ba7-b43d-60191e9957fa admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:53.740212 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99734b-d164-4ba7-b43d-60191e9957fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:53.740489 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99734b-d164-4ba7-b43d-60191e9957fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:53.741770 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99734b-d164-4ba7-b43d-60191e9957fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:53.744376 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99734b-d164-4ba7-b43d-60191e9957fa admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:54.772305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 922/1832] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:05:53 2022] POST /identity/v3/projects => generated 387 bytes in 1137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:54.774414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99734b-d164-4ba7-b43d-60191e9957fa admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:54.783736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1079f87a-6bd3-4d45-9bda-26e17aed8a86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8672fc0906f84bf79d2b41ea2705883c'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:54.787300 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1079f87a-6bd3-4d45-9bda-26e17aed8a86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8672fc0906f84bf79d2b41ea2705883c'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:54.791583 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1079f87a-6bd3-4d45-9bda-26e17aed8a86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8672fc0906f84bf79d2b41ea2705883c'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:54.804188 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1079f87a-6bd3-4d45-9bda-26e17aed8a86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8672fc0906f84bf79d2b41ea2705883c'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:54.804894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1079f87a-6bd3-4d45-9bda-26e17aed8a86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8672fc0906f84bf79d2b41ea2705883c'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:54.840119 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99734b-d164-4ba7-b43d-60191e9957fa admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:54.844848 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be9165-2550-46e2-9298-98f2c4cc8728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:54.845493 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be9165-2550-46e2-9298-98f2c4cc8728 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:54.845777 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be9165-2550-46e2-9298-98f2c4cc8728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:54.846053 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be9165-2550-46e2-9298-98f2c4cc8728 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:54.846734 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be9165-2550-46e2-9298-98f2c4cc8728 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:54.848716 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be9165-2550-46e2-9298-98f2c4cc8728 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:54.870186 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:54.933605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 911/1833] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:05:53 2022] DELETE /identity/v3/users/8672fc0906f84bf79d2b41ea2705883c => generated 0 bytes in 1680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:54.964785 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 923/1834] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:05:54 2022] POST /identity/v3/users => generated 300 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:05:54.969391 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1079f87a-6bd3-4d45-9bda-26e17aed8a86 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:54.970821 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-1079f87a-6bd3-4d45-9bda-26e17aed8a86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:55.073867 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1079f87a-6bd3-4d45-9bda-26e17aed8a86 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:55.081080 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f8a9bd-4bd1-4d92-8d8b-bc8f4cf041ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:55.081721 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8a9bd-4bd1-4d92-8d8b-bc8f4cf041ba admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:55.082020 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8a9bd-4bd1-4d92-8d8b-bc8f4cf041ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:55.082292 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8a9bd-4bd1-4d92-8d8b-bc8f4cf041ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:55.082974 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f8a9bd-4bd1-4d92-8d8b-bc8f4cf041ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:55.084701 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f8a9bd-4bd1-4d92-8d8b-bc8f4cf041ba admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:55.095674 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 912/1835] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:55.100873 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be9165-2550-46e2-9298-98f2c4cc8728 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:55.113813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be9165-2550-46e2-9298-98f2c4cc8728 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:55.121005 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a42b05-f62c-4129-a4c4-143b95291d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:55.121594 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a42b05-f62c-4129-a4c4-143b95291d56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:55.121872 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a42b05-f62c-4129-a4c4-143b95291d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:55.122143 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a42b05-f62c-4129-a4c4-143b95291d56 admin admin] PATH_INFO: `/v3/projects/6ffd4e04cb794d58be2375b023f3ae8f/users/e7f84d3eb63c4e7b87883de16a99db9c/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:55.142802 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a42b05-f62c-4129-a4c4-143b95291d56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ffd4e04cb794d58be2375b023f3ae8f, user_id=e7f84d3eb63c4e7b87883de16a99db9c, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:55.153607 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a42b05-f62c-4129-a4c4-143b95291d56 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:55.276551 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 924/1836] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:55 2022] PUT /identity/v3/projects/6ffd4e04cb794d58be2375b023f3ae8f/users/e7f84d3eb63c4e7b87883de16a99db9c/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:55.280753 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f8a9bd-4bd1-4d92-8d8b-bc8f4cf041ba admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:55.292178 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f8a9bd-4bd1-4d92-8d8b-bc8f4cf041ba admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:55.353362 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9980b-119d-4bb5-9095-e9866f5baf4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:55.353965 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9980b-119d-4bb5-9095-e9866f5baf4a admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:55.354245 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9980b-119d-4bb5-9095-e9866f5baf4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:55.354515 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9980b-119d-4bb5-9095-e9866f5baf4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:55.355149 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9980b-119d-4bb5-9095-e9866f5baf4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:55.356784 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9980b-119d-4bb5-9095-e9866f5baf4a admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:55.368163 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 913/1837] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:55 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:05:55.370596 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a42b05-f62c-4129-a4c4-143b95291d56 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:55.381109 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a42b05-f62c-4129-a4c4-143b95291d56 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:55.390303 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007f10d-9c3f-4995-bcc0-560ebae83388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:55.390966 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007f10d-9c3f-4995-bcc0-560ebae83388 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:55.391241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007f10d-9c3f-4995-bcc0-560ebae83388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:55.391509 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007f10d-9c3f-4995-bcc0-560ebae83388 admin admin] PATH_INFO: `/v3/projects/6ffd4e04cb794d58be2375b023f3ae8f/users/e7f84d3eb63c4e7b87883de16a99db9c/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:55.393491 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007f10d-9c3f-4995-bcc0-560ebae83388 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ffd4e04cb794d58be2375b023f3ae8f, user_id=e7f84d3eb63c4e7b87883de16a99db9c, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:55.396148 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007f10d-9c3f-4995-bcc0-560ebae83388 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:55.407594 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 925/1838] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:55 2022] PUT /identity/v3/projects/6ffd4e04cb794d58be2375b023f3ae8f/users/e7f84d3eb63c4e7b87883de16a99db9c/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:55.422033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6b53a-8d35-46d6-856d-20f6479dcd56 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:55.422698 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6b53a-8d35-46d6-856d-20f6479dcd56 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:55.423041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6b53a-8d35-46d6-856d-20f6479dcd56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:55.451384 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007f10d-9c3f-4995-bcc0-560ebae83388 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:55.452716 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c007f10d-9c3f-4995-bcc0-560ebae83388 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:55.456102 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-05a6b53a-8d35-46d6-856d-20f6479dcd56 None None] MFA Rules not processed for user `e7f84d3eb63c4e7b87883de16a99db9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:55.457305 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-05a6b53a-8d35-46d6-856d-20f6479dcd56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:55.534066 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007f10d-9c3f-4995-bcc0-560ebae83388 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:55.539769 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 914/1839] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:55 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:55.539925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a58f70-8982-4c88-951d-56519b88e0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:55.540565 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a58f70-8982-4c88-951d-56519b88e0f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:55.540843 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a58f70-8982-4c88-951d-56519b88e0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:55.541114 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a58f70-8982-4c88-951d-56519b88e0f2 admin admin] PATH_INFO: `/v3/projects/133eb16a74d0449f9c734b0cd4aaf3cc` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:55.544104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a58f70-8982-4c88-951d-56519b88e0f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=133eb16a74d0449f9c734b0cd4aaf3cc)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:55.546044 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a58f70-8982-4c88-951d-56519b88e0f2 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:55.555758 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6b53a-8d35-46d6-856d-20f6479dcd56 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:55.558223 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-05a6b53a-8d35-46d6-856d-20f6479dcd56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:55.619665 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6b53a-8d35-46d6-856d-20f6479dcd56 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:55.623266 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13a482-6b30-4623-ad9e-356ed2b65742 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:55.623863 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13a482-6b30-4623-ad9e-356ed2b65742 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:55.624177 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13a482-6b30-4623-ad9e-356ed2b65742 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:55.624454 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13a482-6b30-4623-ad9e-356ed2b65742 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:55.634072 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d13a482-6b30-4623-ad9e-356ed2b65742 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:55.635439 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d13a482-6b30-4623-ad9e-356ed2b65742 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:55.644893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 915/1840] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:55 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:05:55.671069 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c3a58f70-8982-4c88-951d-56519b88e0f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '133eb16a74d0449f9c734b0cd4aaf3cc', 'request_id': 'req-c3a58f70-8982-4c88-951d-56519b88e0f2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:55.677516 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c3a58f70-8982-4c88-951d-56519b88e0f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '133eb16a74d0449f9c734b0cd4aaf3cc', 'request_id': 'req-c3a58f70-8982-4c88-951d-56519b88e0f2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:55.756230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c3a58f70-8982-4c88-951d-56519b88e0f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '133eb16a74d0449f9c734b0cd4aaf3cc', 'request_id': 'req-c3a58f70-8982-4c88-951d-56519b88e0f2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:55.771049 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c3a58f70-8982-4c88-951d-56519b88e0f2 admin admin] The token cache is being invalidate because project 133eb16a74d0449f9c734b0cd4aaf3cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:05:55.771716 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c3a58f70-8982-4c88-951d-56519b88e0f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:55.772137 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c3a58f70-8982-4c88-951d-56519b88e0f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:55.778652 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5404e-6a67-49f6-8b66-b000af0c77fb None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:55.778937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5404e-6a67-49f6-8b66-b000af0c77fb None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:55.779397 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5404e-6a67-49f6-8b66-b000af0c77fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:55.780154 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 926/1841] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:05:55 2022] DELETE /identity/v3/projects/133eb16a74d0449f9c734b0cd4aaf3cc => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:05:55.801555 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-68d5404e-6a67-49f6-8b66-b000af0c77fb None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:55.802770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-68d5404e-6a67-49f6-8b66-b000af0c77fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:55.814321 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:55.870751 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 916/1842] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:55.878611 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16e04c-b0a2-4ec9-bbd6-1d1f8fe82525 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:55.878901 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16e04c-b0a2-4ec9-bbd6-1d1f8fe82525 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:55.879179 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16e04c-b0a2-4ec9-bbd6-1d1f8fe82525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:55.880849 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:55.906473 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-4d16e04c-b0a2-4ec9-bbd6-1d1f8fe82525 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:55.907712 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4d16e04c-b0a2-4ec9-bbd6-1d1f8fe82525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:55.916456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 927/1843] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:55.925998 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d5404e-6a67-49f6-8b66-b000af0c77fb None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:55.936159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d5404e-6a67-49f6-8b66-b000af0c77fb None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:55.941027 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16a02f-857a-4137-8c10-c731442b343e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:55.941643 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16a02f-857a-4137-8c10-c731442b343e admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:55.941931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16a02f-857a-4137-8c10-c731442b343e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:55.942210 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16a02f-857a-4137-8c10-c731442b343e admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:55.942881 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16a02f-857a-4137-8c10-c731442b343e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:05:55.943173 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16a02f-857a-4137-8c10-c731442b343e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:55.944880 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16a02f-857a-4137-8c10-c731442b343e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:55.957153 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 917/1844] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:05:55.961634 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16e04c-b0a2-4ec9-bbd6-1d1f8fe82525 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:55.970648 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16e04c-b0a2-4ec9-bbd6-1d1f8fe82525 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:55.976689 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a47e5c-6576-4e6d-8d7a-be97ef3d59a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:55.976998 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a47e5c-6576-4e6d-8d7a-be97ef3d59a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:55.977281 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a47e5c-6576-4e6d-8d7a-be97ef3d59a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:55.977549 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a47e5c-6576-4e6d-8d7a-be97ef3d59a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:55.978195 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a47e5c-6576-4e6d-8d7a-be97ef3d59a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:55.980298 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a47e5c-6576-4e6d-8d7a-be97ef3d59a0 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:56.097209 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 928/1845] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:05:55 2022] POST /identity/v3/projects => generated 387 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:56.101257 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16a02f-857a-4137-8c10-c731442b343e admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:56.111518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16a02f-857a-4137-8c10-c731442b343e admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:56.179248 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795adb8-1369-42d1-8a5f-f757d66fbc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:56.181327 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795adb8-1369-42d1-8a5f-f757d66fbc17 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:56.181327 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795adb8-1369-42d1-8a5f-f757d66fbc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:56.181327 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795adb8-1369-42d1-8a5f-f757d66fbc17 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:56.181327 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6795adb8-1369-42d1-8a5f-f757d66fbc17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:56.183060 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6795adb8-1369-42d1-8a5f-f757d66fbc17 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:56.258317 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 918/1846] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:05:56 2022] POST /identity/v3/users => generated 294 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:05:56.262395 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a47e5c-6576-4e6d-8d7a-be97ef3d59a0 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:56.274079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a47e5c-6576-4e6d-8d7a-be97ef3d59a0 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:56.278714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7d9b6-927b-4cc8-b26f-fcea9ac93229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:56.279310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7d9b6-927b-4cc8-b26f-fcea9ac93229 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:56.279606 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7d9b6-927b-4cc8-b26f-fcea9ac93229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:56.279892 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7d9b6-927b-4cc8-b26f-fcea9ac93229 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:56.280576 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7d9b6-927b-4cc8-b26f-fcea9ac93229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:56.282220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7d9b6-927b-4cc8-b26f-fcea9ac93229 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:56.292634 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 929/1847] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:56.299531 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795adb8-1369-42d1-8a5f-f757d66fbc17 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:56.310182 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795adb8-1369-42d1-8a5f-f757d66fbc17 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:56.315060 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ca088-b124-434b-aa65-a9f5f8c67b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:56.315690 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ca088-b124-434b-aa65-a9f5f8c67b01 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:56.315969 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ca088-b124-434b-aa65-a9f5f8c67b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:56.316283 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ca088-b124-434b-aa65-a9f5f8c67b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:56.316955 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84ca088-b124-434b-aa65-a9f5f8c67b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:56.318611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84ca088-b124-434b-aa65-a9f5f8c67b01 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:56.331390 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 919/1848] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:56.336285 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7d9b6-927b-4cc8-b26f-fcea9ac93229 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:56.347068 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7d9b6-927b-4cc8-b26f-fcea9ac93229 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:56.352772 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab83e7-78ba-47ab-b7eb-ec7da1e0bd32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:56.353415 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab83e7-78ba-47ab-b7eb-ec7da1e0bd32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:56.353697 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab83e7-78ba-47ab-b7eb-ec7da1e0bd32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:56.353990 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab83e7-78ba-47ab-b7eb-ec7da1e0bd32 admin admin] PATH_INFO: `/v3/projects/adee4d1357ca45d18b4d7f3ac03a3c7c/users/fa01baf61ba94c03b40234b0f6017ebc/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:56.371483 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab83e7-78ba-47ab-b7eb-ec7da1e0bd32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adee4d1357ca45d18b4d7f3ac03a3c7c, user_id=fa01baf61ba94c03b40234b0f6017ebc, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:56.374562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab83e7-78ba-47ab-b7eb-ec7da1e0bd32 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:56.463280 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 930/1849] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:56 2022] PUT /identity/v3/projects/adee4d1357ca45d18b4d7f3ac03a3c7c/users/fa01baf61ba94c03b40234b0f6017ebc/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:56.475498 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0e30d-3898-408e-8ea9-ad8723228724 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:56.475780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0e30d-3898-408e-8ea9-ad8723228724 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:56.476096 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0e30d-3898-408e-8ea9-ad8723228724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:56.503989 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-e6f0e30d-3898-408e-8ea9-ad8723228724 None None] MFA Rules not processed for user `fa01baf61ba94c03b40234b0f6017ebc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:56.505272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e6f0e30d-3898-408e-8ea9-ad8723228724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:56.566660 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 920/1850] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:56.571747 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab83e7-78ba-47ab-b7eb-ec7da1e0bd32 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:56.636738 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab83e7-78ba-47ab-b7eb-ec7da1e0bd32 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:56.641513 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c75457-190f-4c9b-b954-98937505fb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:56.642192 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c75457-190f-4c9b-b954-98937505fb49 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:56.642469 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c75457-190f-4c9b-b954-98937505fb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:56.642739 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c75457-190f-4c9b-b954-98937505fb49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:56.643375 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c75457-190f-4c9b-b954-98937505fb49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:56.645327 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c75457-190f-4c9b-b954-98937505fb49 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:56.679779 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:56.745957 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 931/1851] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:05:56 2022] POST /identity/v3/projects => generated 387 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:56.755909 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0e30d-3898-408e-8ea9-ad8723228724 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:56.775794 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0e30d-3898-408e-8ea9-ad8723228724 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:56.803490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c75457-190f-4c9b-b954-98937505fb49 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:56.805974 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-26c75457-190f-4c9b-b954-98937505fb49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:56.841195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b5b4f-56a3-4380-896d-d52f972331f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:56.841816 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b5b4f-56a3-4380-896d-d52f972331f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:56.842094 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b5b4f-56a3-4380-896d-d52f972331f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:56.842360 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b5b4f-56a3-4380-896d-d52f972331f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:56.842993 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b5b4f-56a3-4380-896d-d52f972331f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:56.845858 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b5b4f-56a3-4380-896d-d52f972331f3 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:56.894389 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c75457-190f-4c9b-b954-98937505fb49 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:56.894389 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c00872-71e3-498e-bc01-035344fd64db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:56.894389 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c00872-71e3-498e-bc01-035344fd64db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:56.894389 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c00872-71e3-498e-bc01-035344fd64db admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:56.895354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c00872-71e3-498e-bc01-035344fd64db admin admin] PATH_INFO: `/v3/users/364798a6a1e14c44b68f63851927c014` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:56.895354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c00872-71e3-498e-bc01-035344fd64db admin admin] RBAC: Authorizing `identity:delete_user(user_id=364798a6a1e14c44b68f63851927c014)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:56.895354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c00872-71e3-498e-bc01-035344fd64db admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:56.967545 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 921/1852] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:05:56 2022] POST /identity/v3/users => generated 300 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:05:56.968437 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b5b4f-56a3-4380-896d-d52f972331f3 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:56.970079 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d64b5b4f-56a3-4380-896d-d52f972331f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:57.033695 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b5b4f-56a3-4380-896d-d52f972331f3 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:57.037469 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e32bc-59a8-43cd-a692-3aee2d2365b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:57.038600 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e32bc-59a8-43cd-a692-3aee2d2365b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:57.038600 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e32bc-59a8-43cd-a692-3aee2d2365b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:57.038600 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e32bc-59a8-43cd-a692-3aee2d2365b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:57.041522 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:57.042649 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0c3e32bc-59a8-43cd-a692-3aee2d2365b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:57.085166 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c0c00872-71e3-498e-bc01-035344fd64db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '364798a6a1e14c44b68f63851927c014'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:57.085837 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c0c00872-71e3-498e-bc01-035344fd64db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '364798a6a1e14c44b68f63851927c014'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:57.086481 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c0c00872-71e3-498e-bc01-035344fd64db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '364798a6a1e14c44b68f63851927c014'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:57.093297 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c0c00872-71e3-498e-bc01-035344fd64db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '364798a6a1e14c44b68f63851927c014'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:57.105317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c0c00872-71e3-498e-bc01-035344fd64db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '364798a6a1e14c44b68f63851927c014'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:57.117872 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e32bc-59a8-43cd-a692-3aee2d2365b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:57.119456 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e32bc-59a8-43cd-a692-3aee2d2365b4 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:57.129537 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 922/1853] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:57.130391 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e32bc-59a8-43cd-a692-3aee2d2365b4 service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:57.141358 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e32bc-59a8-43cd-a692-3aee2d2365b4 service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:57.146099 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf05d0-f49d-4dd9-a824-f9d542c5df3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:57.148025 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf05d0-f49d-4dd9-a824-f9d542c5df3e admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:57.148025 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf05d0-f49d-4dd9-a824-f9d542c5df3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:57.148025 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf05d0-f49d-4dd9-a824-f9d542c5df3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:57.148025 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf05d0-f49d-4dd9-a824-f9d542c5df3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:57.149546 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf05d0-f49d-4dd9-a824-f9d542c5df3e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:57.159888 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 923/1854] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:05:57.163967 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf05d0-f49d-4dd9-a824-f9d542c5df3e admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:57.165692 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-90bf05d0-f49d-4dd9-a824-f9d542c5df3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:57.177990 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf05d0-f49d-4dd9-a824-f9d542c5df3e admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:57.183433 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52bd021-3a06-4981-803f-669af768f012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:57.184124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52bd021-3a06-4981-803f-669af768f012 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:57.184416 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52bd021-3a06-4981-803f-669af768f012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:57.184694 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52bd021-3a06-4981-803f-669af768f012 admin admin] PATH_INFO: `/v3/projects/a316c02012534eca8a4ef12b1873e7e2/users/efa33827a7104939aa297a7645b6c14a/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:57.202663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52bd021-3a06-4981-803f-669af768f012 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a316c02012534eca8a4ef12b1873e7e2, user_id=efa33827a7104939aa297a7645b6c14a, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:57.205501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52bd021-3a06-4981-803f-669af768f012 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:57.207401 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 932/1855] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:05:56 2022] DELETE /identity/v3/users/364798a6a1e14c44b68f63851927c014 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:05:57.208827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5636d-62c7-4cac-9d3d-3aa458f381ca None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:57.209189 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5636d-62c7-4cac-9d3d-3aa458f381ca None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:57.209471 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5636d-62c7-4cac-9d3d-3aa458f381ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:57.234722 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-81a5636d-62c7-4cac-9d3d-3aa458f381ca None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:57.234722 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-81a5636d-62c7-4cac-9d3d-3aa458f381ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:57.292551 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 924/1856] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:57 2022] PUT /identity/v3/projects/a316c02012534eca8a4ef12b1873e7e2/users/efa33827a7104939aa297a7645b6c14a/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:05:57.294388 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 933/1857] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:57.296312 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52bd021-3a06-4981-803f-669af768f012 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:57.299856 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:57.305263 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5636d-62c7-4cac-9d3d-3aa458f381ca None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:57.306605 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-81a5636d-62c7-4cac-9d3d-3aa458f381ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:57.362138 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52bd021-3a06-4981-803f-669af768f012 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:57.362376 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5636d-62c7-4cac-9d3d-3aa458f381ca None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:57.367851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ff914-476e-4417-b4cb-645ca160b6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:57.368488 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ff914-476e-4417-b4cb-645ca160b6a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:57.368766 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ff914-476e-4417-b4cb-645ca160b6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:57.369036 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ff914-476e-4417-b4cb-645ca160b6a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:57.369715 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ff914-476e-4417-b4cb-645ca160b6a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:57.371360 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ff914-476e-4417-b4cb-645ca160b6a2 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:57.381837 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 925/1858] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:05:57 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:05:57.386290 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ff914-476e-4417-b4cb-645ca160b6a2 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:57.396200 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ff914-476e-4417-b4cb-645ca160b6a2 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:57.400956 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1234d7-1141-46cc-9c0d-fc7b042d4e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:57.401610 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1234d7-1141-46cc-9c0d-fc7b042d4e22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:57.401942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1234d7-1141-46cc-9c0d-fc7b042d4e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:57.402217 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1234d7-1141-46cc-9c0d-fc7b042d4e22 admin admin] PATH_INFO: `/v3/projects/a316c02012534eca8a4ef12b1873e7e2/users/efa33827a7104939aa297a7645b6c14a/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:57.404249 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1234d7-1141-46cc-9c0d-fc7b042d4e22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a316c02012534eca8a4ef12b1873e7e2, user_id=efa33827a7104939aa297a7645b6c14a, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:57.406926 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1234d7-1141-46cc-9c0d-fc7b042d4e22 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:57.413578 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e0359f-574f-4da4-86aa-9e72ba8ffdcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:57.414230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e0359f-574f-4da4-86aa-9e72ba8ffdcc service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:57.414510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e0359f-574f-4da4-86aa-9e72ba8ffdcc service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:57.414778 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e0359f-574f-4da4-86aa-9e72ba8ffdcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:57.418684 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 926/1859] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:05:57 2022] PUT /identity/v3/projects/a316c02012534eca8a4ef12b1873e7e2/users/efa33827a7104939aa297a7645b6c14a/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:05:57.424595 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e0359f-574f-4da4-86aa-9e72ba8ffdcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:57.426033 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e0359f-574f-4da4-86aa-9e72ba8ffdcc service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:57.429528 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502cd86-3666-4e71-97f3-59d4eb923e9c None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:57.429811 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502cd86-3666-4e71-97f3-59d4eb923e9c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:57.430081 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502cd86-3666-4e71-97f3-59d4eb923e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:57.437363 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 934/1860] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:05:57.458579 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-3502cd86-3666-4e71-97f3-59d4eb923e9c None None] MFA Rules not processed for user `efa33827a7104939aa297a7645b6c14a`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:57.460007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3502cd86-3666-4e71-97f3-59d4eb923e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:57.485461 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:57.531443 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 927/1861] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:57 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:57.543739 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e0359f-574f-4da4-86aa-9e72ba8ffdcc service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:57.545331 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-13e0359f-574f-4da4-86aa-9e72ba8ffdcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:57.603999 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e0359f-574f-4da4-86aa-9e72ba8ffdcc service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:57.654829 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7596eca-c07a-4d57-b805-c460ed146262 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:57.655676 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7596eca-c07a-4d57-b805-c460ed146262 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:57.656016 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7596eca-c07a-4d57-b805-c460ed146262 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:57.656320 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7596eca-c07a-4d57-b805-c460ed146262 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:57.666449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7596eca-c07a-4d57-b805-c460ed146262 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:57.667924 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7596eca-c07a-4d57-b805-c460ed146262 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:57.677925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 935/1862] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:57 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:05:57.802598 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502cd86-3666-4e71-97f3-59d4eb923e9c None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:57.820579 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:57.870827 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502cd86-3666-4e71-97f3-59d4eb923e9c None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:57.875994 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66055fc6-decd-46ce-9122-4ce747987499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:57.876675 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66055fc6-decd-46ce-9122-4ce747987499 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:57.877333 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66055fc6-decd-46ce-9122-4ce747987499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:57.877681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66055fc6-decd-46ce-9122-4ce747987499 admin admin] PATH_INFO: `/v3/users/fa01baf61ba94c03b40234b0f6017ebc` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:57.883623 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66055fc6-decd-46ce-9122-4ce747987499 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa01baf61ba94c03b40234b0f6017ebc)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:57.885163 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7596eca-c07a-4d57-b805-c460ed146262 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:57.885789 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66055fc6-decd-46ce-9122-4ce747987499 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:57.888258 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c7596eca-c07a-4d57-b805-c460ed146262 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:57.929760 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7596eca-c07a-4d57-b805-c460ed146262 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:57.936775 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145046e-11ef-411f-891e-11b64ece75e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:57.937764 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145046e-11ef-411f-891e-11b64ece75e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:57.938055 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145046e-11ef-411f-891e-11b64ece75e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:57.938376 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145046e-11ef-411f-891e-11b64ece75e3 admin admin] PATH_INFO: `/v3/projects/70a8e6a050654249b3f33179c225947b` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:57.939650 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145046e-11ef-411f-891e-11b64ece75e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70a8e6a050654249b3f33179c225947b)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:57.941822 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145046e-11ef-411f-891e-11b64ece75e3 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:58.123670 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-66055fc6-decd-46ce-9122-4ce747987499 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa01baf61ba94c03b40234b0f6017ebc'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:58.124365 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-66055fc6-decd-46ce-9122-4ce747987499 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa01baf61ba94c03b40234b0f6017ebc'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:58.129882 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e145046e-11ef-411f-891e-11b64ece75e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70a8e6a050654249b3f33179c225947b', 'request_id': 'req-e145046e-11ef-411f-891e-11b64ece75e3'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:58.133619 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-66055fc6-decd-46ce-9122-4ce747987499 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa01baf61ba94c03b40234b0f6017ebc'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:58.137750 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e145046e-11ef-411f-891e-11b64ece75e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70a8e6a050654249b3f33179c225947b', 'request_id': 'req-e145046e-11ef-411f-891e-11b64ece75e3'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:58.149966 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-66055fc6-decd-46ce-9122-4ce747987499 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa01baf61ba94c03b40234b0f6017ebc'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:58.150718 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-66055fc6-decd-46ce-9122-4ce747987499 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa01baf61ba94c03b40234b0f6017ebc'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:58.219110 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:58.220100 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e145046e-11ef-411f-891e-11b64ece75e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70a8e6a050654249b3f33179c225947b', 'request_id': 'req-e145046e-11ef-411f-891e-11b64ece75e3'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:58.231255 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e145046e-11ef-411f-891e-11b64ece75e3 admin admin] The token cache is being invalidate because project 70a8e6a050654249b3f33179c225947b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:05:58.231550 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e145046e-11ef-411f-891e-11b64ece75e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:58.232285 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e145046e-11ef-411f-891e-11b64ece75e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:58.233779 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 936/1863] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:05:57 2022] DELETE /identity/v3/projects/70a8e6a050654249b3f33179c225947b => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:05:58.237596 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145046e-11ef-411f-891e-11b64ece75e3 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:58.238971 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e145046e-11ef-411f-891e-11b64ece75e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:58.302606 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145046e-11ef-411f-891e-11b64ece75e3 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:58.307653 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00464e2f-6e1c-4f94-b93f-95f46b0dbe9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:58.308385 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00464e2f-6e1c-4f94-b93f-95f46b0dbe9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:58.308690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00464e2f-6e1c-4f94-b93f-95f46b0dbe9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:58.308965 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00464e2f-6e1c-4f94-b93f-95f46b0dbe9b admin admin] PATH_INFO: `/v3/users/323ef20684234326b4f31fb223fef957` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:58.310234 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00464e2f-6e1c-4f94-b93f-95f46b0dbe9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=323ef20684234326b4f31fb223fef957)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:58.312238 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00464e2f-6e1c-4f94-b93f-95f46b0dbe9b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:58.325551 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 928/1864] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:05:57 2022] DELETE /identity/v3/users/fa01baf61ba94c03b40234b0f6017ebc => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:58.327308 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790654f-ca51-49e4-b184-323e41df5b17 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:58.327595 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790654f-ca51-49e4-b184-323e41df5b17 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:58.327870 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790654f-ca51-49e4-b184-323e41df5b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:58.348244 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-e790654f-ca51-49e4-b184-323e41df5b17 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:58.349594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e790654f-ca51-49e4-b184-323e41df5b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:58.409076 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 929/1865] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:05:58.419193 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e790654f-ca51-49e4-b184-323e41df5b17 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:58.420521 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e790654f-ca51-49e4-b184-323e41df5b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:58.439512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:58.480661 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e790654f-ca51-49e4-b184-323e41df5b17 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:58.484298 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb82ac1-d58e-475d-bbab-addda1e1b910 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:58.488524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb82ac1-d58e-475d-bbab-addda1e1b910 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:58.488524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb82ac1-d58e-475d-bbab-addda1e1b910 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:58.488524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb82ac1-d58e-475d-bbab-addda1e1b910 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:58.494272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb82ac1-d58e-475d-bbab-addda1e1b910 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:58.495703 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb82ac1-d58e-475d-bbab-addda1e1b910 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:58.505426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 930/1866] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:05:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:05:58.522654 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-00464e2f-6e1c-4f94-b93f-95f46b0dbe9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '323ef20684234326b4f31fb223fef957'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:58.523334 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-00464e2f-6e1c-4f94-b93f-95f46b0dbe9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '323ef20684234326b4f31fb223fef957'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:58.523948 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-00464e2f-6e1c-4f94-b93f-95f46b0dbe9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '323ef20684234326b4f31fb223fef957'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:58.531112 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-00464e2f-6e1c-4f94-b93f-95f46b0dbe9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '323ef20684234326b4f31fb223fef957'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:58.543501 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-00464e2f-6e1c-4f94-b93f-95f46b0dbe9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '323ef20684234326b4f31fb223fef957'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:58.639254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 937/1867] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:05:58 2022] DELETE /identity/v3/users/323ef20684234326b4f31fb223fef957 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:58.975219 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb82ac1-d58e-475d-bbab-addda1e1b910 service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:58.977596 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5eb82ac1-d58e-475d-bbab-addda1e1b910 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:58.994995 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb82ac1-d58e-475d-bbab-addda1e1b910 service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:59.062379 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e656b48-6412-47f8-b11e-7c62d74a45e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:59.062940 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e656b48-6412-47f8-b11e-7c62d74a45e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:59.063218 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e656b48-6412-47f8-b11e-7c62d74a45e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:59.063488 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e656b48-6412-47f8-b11e-7c62d74a45e1 admin admin] PATH_INFO: `/v3/projects/adee4d1357ca45d18b4d7f3ac03a3c7c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:59.064675 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e656b48-6412-47f8-b11e-7c62d74a45e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adee4d1357ca45d18b4d7f3ac03a3c7c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:59.068416 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e656b48-6412-47f8-b11e-7c62d74a45e1 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:59.191877 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-4e656b48-6412-47f8-b11e-7c62d74a45e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adee4d1357ca45d18b4d7f3ac03a3c7c', 'request_id': 'req-4e656b48-6412-47f8-b11e-7c62d74a45e1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:59.262854 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-4e656b48-6412-47f8-b11e-7c62d74a45e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adee4d1357ca45d18b4d7f3ac03a3c7c', 'request_id': 'req-4e656b48-6412-47f8-b11e-7c62d74a45e1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:59.263760 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00464e2f-6e1c-4f94-b93f-95f46b0dbe9b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:59.265248 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-00464e2f-6e1c-4f94-b93f-95f46b0dbe9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:59.272397 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-4e656b48-6412-47f8-b11e-7c62d74a45e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adee4d1357ca45d18b4d7f3ac03a3c7c', 'request_id': 'req-4e656b48-6412-47f8-b11e-7c62d74a45e1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:59.279216 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00464e2f-6e1c-4f94-b93f-95f46b0dbe9b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:59.296876 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-4e656b48-6412-47f8-b11e-7c62d74a45e1 admin admin] The token cache is being invalidate because project adee4d1357ca45d18b4d7f3ac03a3c7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:05:59.297398 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-4e656b48-6412-47f8-b11e-7c62d74a45e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:59.298855 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-4e656b48-6412-47f8-b11e-7c62d74a45e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:59.304922 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 931/1868] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:05:58 2022] DELETE /identity/v3/projects/adee4d1357ca45d18b4d7f3ac03a3c7c => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:59.307078 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e656b48-6412-47f8-b11e-7c62d74a45e1 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:59.309934 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-4e656b48-6412-47f8-b11e-7c62d74a45e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:59.344864 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d40566d-6b1b-4fd8-ab2d-97a6b7ef2649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:59.345684 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40566d-6b1b-4fd8-ab2d-97a6b7ef2649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:59.345992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40566d-6b1b-4fd8-ab2d-97a6b7ef2649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:59.346337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40566d-6b1b-4fd8-ab2d-97a6b7ef2649 admin admin] PATH_INFO: `/v3/projects/c63fec25e1184800b7d25bc187b2b063` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:59.347558 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d40566d-6b1b-4fd8-ab2d-97a6b7ef2649 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c63fec25e1184800b7d25bc187b2b063)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:59.349712 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d40566d-6b1b-4fd8-ab2d-97a6b7ef2649 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:59.386502 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:59.423092 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:05:59.446521 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e656b48-6412-47f8-b11e-7c62d74a45e1 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:59.454959 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9975c-b43e-4797-b42a-a82a39b62463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:59.455693 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9975c-b43e-4797-b42a-a82a39b62463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:59.455973 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9975c-b43e-4797-b42a-a82a39b62463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:59.456335 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9975c-b43e-4797-b42a-a82a39b62463 admin admin] PATH_INFO: `/v3/users/efa33827a7104939aa297a7645b6c14a` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:59.458698 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9975c-b43e-4797-b42a-a82a39b62463 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efa33827a7104939aa297a7645b6c14a)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:59.459886 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9975c-b43e-4797-b42a-a82a39b62463 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:59.494972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9d40566d-6b1b-4fd8-ab2d-97a6b7ef2649 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c63fec25e1184800b7d25bc187b2b063', 'request_id': 'req-9d40566d-6b1b-4fd8-ab2d-97a6b7ef2649'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:59.503167 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9d40566d-6b1b-4fd8-ab2d-97a6b7ef2649 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c63fec25e1184800b7d25bc187b2b063', 'request_id': 'req-9d40566d-6b1b-4fd8-ab2d-97a6b7ef2649'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:59.589526 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9d40566d-6b1b-4fd8-ab2d-97a6b7ef2649 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c63fec25e1184800b7d25bc187b2b063', 'request_id': 'req-9d40566d-6b1b-4fd8-ab2d-97a6b7ef2649'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:59.601497 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9d40566d-6b1b-4fd8-ab2d-97a6b7ef2649 admin admin] The token cache is being invalidate because project c63fec25e1184800b7d25bc187b2b063 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:05:59.601901 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9d40566d-6b1b-4fd8-ab2d-97a6b7ef2649 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:59.602567 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9d40566d-6b1b-4fd8-ab2d-97a6b7ef2649 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:59.604207 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 938/1869] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:05:59 2022] DELETE /identity/v3/projects/c63fec25e1184800b7d25bc187b2b063 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:05:59.608929 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e517ca-78ce-4d27-880d-abaf447108dd None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:59.609218 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e517ca-78ce-4d27-880d-abaf447108dd None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:59.609499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e517ca-78ce-4d27-880d-abaf447108dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:59.629276 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a5e517ca-78ce-4d27-880d-abaf447108dd None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:59.630480 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a5e517ca-78ce-4d27-880d-abaf447108dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:59.673399 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2df9975c-b43e-4797-b42a-a82a39b62463 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efa33827a7104939aa297a7645b6c14a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:59.674055 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2df9975c-b43e-4797-b42a-a82a39b62463 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efa33827a7104939aa297a7645b6c14a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:59.681697 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2df9975c-b43e-4797-b42a-a82a39b62463 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efa33827a7104939aa297a7645b6c14a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:59.691063 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 939/1870] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:05:59.693914 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2df9975c-b43e-4797-b42a-a82a39b62463 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efa33827a7104939aa297a7645b6c14a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:59.694648 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2df9975c-b43e-4797-b42a-a82a39b62463 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efa33827a7104939aa297a7645b6c14a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:05:59.699908 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c55b75-d24a-4723-843f-c5dd8df913d6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:59.700301 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c55b75-d24a-4723-843f-c5dd8df913d6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:59.700647 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c55b75-d24a-4723-843f-c5dd8df913d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:59.720014 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a2c55b75-d24a-4723-843f-c5dd8df913d6 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:05:59.721339 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a2c55b75-d24a-4723-843f-c5dd8df913d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:05:59.729943 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 940/1871] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:05:59.733933 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c55b75-d24a-4723-843f-c5dd8df913d6 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:59.743342 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c55b75-d24a-4723-843f-c5dd8df913d6 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:59.747940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a9971-8cf2-45ed-88bc-acfeba982d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:59.748627 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a9971-8cf2-45ed-88bc-acfeba982d8d admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:59.748861 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a9971-8cf2-45ed-88bc-acfeba982d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:59.749137 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a9971-8cf2-45ed-88bc-acfeba982d8d admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:59.749907 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a9971-8cf2-45ed-88bc-acfeba982d8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:05:59.750217 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a9971-8cf2-45ed-88bc-acfeba982d8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:59.751806 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a9971-8cf2-45ed-88bc-acfeba982d8d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:59.763623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 941/1872] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:05:59.767867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a9971-8cf2-45ed-88bc-acfeba982d8d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:59.772920 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 932/1873] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:05:59 2022] DELETE /identity/v3/users/efa33827a7104939aa297a7645b6c14a => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:05:59.777952 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a9971-8cf2-45ed-88bc-acfeba982d8d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:05:59.839788 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0aa0fc-bea4-42a7-acbf-b9421ae61b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:05:59.840426 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0aa0fc-bea4-42a7-acbf-b9421ae61b98 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:05:59.844240 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0aa0fc-bea4-42a7-acbf-b9421ae61b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:05:59.844240 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0aa0fc-bea4-42a7-acbf-b9421ae61b98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:05:59.844240 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0aa0fc-bea4-42a7-acbf-b9421ae61b98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:05:59.844240 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0aa0fc-bea4-42a7-acbf-b9421ae61b98 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:05:59.934546 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 942/1874] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:05:59 2022] POST /identity/v3/projects => generated 401 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:05:59.941455 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9975c-b43e-4797-b42a-a82a39b62463 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:05:59.951655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9975c-b43e-4797-b42a-a82a39b62463 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:00.018188 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774f575-5741-4413-9534-087dd13f9cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:00.018863 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774f575-5741-4413-9534-087dd13f9cbe admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:00.019145 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774f575-5741-4413-9534-087dd13f9cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:00.019421 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774f575-5741-4413-9534-087dd13f9cbe admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:00.020094 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774f575-5741-4413-9534-087dd13f9cbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:00.022700 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774f575-5741-4413-9534-087dd13f9cbe admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:00.075549 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:00.113208 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 933/1875] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:05:59 2022] POST /identity/v3/users => generated 301 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:00.114416 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0aa0fc-bea4-42a7-acbf-b9421ae61b98 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:00.125133 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0aa0fc-bea4-42a7-acbf-b9421ae61b98 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:00.126983 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:00.130372 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ff246-a7a1-4bce-80d6-2666629a0692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:00.130968 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ff246-a7a1-4bce-80d6-2666629a0692 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:00.131299 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ff246-a7a1-4bce-80d6-2666629a0692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:00.137004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ff246-a7a1-4bce-80d6-2666629a0692 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:00.137004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ff246-a7a1-4bce-80d6-2666629a0692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:00.137004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ff246-a7a1-4bce-80d6-2666629a0692 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:00.144934 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 943/1876] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:00.148930 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774f575-5741-4413-9534-087dd13f9cbe admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:00.158803 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774f575-5741-4413-9534-087dd13f9cbe admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:00.163434 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50594250-760e-4fc2-ba7d-8c2454af4d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:00.168238 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50594250-760e-4fc2-ba7d-8c2454af4d5f admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:00.168238 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50594250-760e-4fc2-ba7d-8c2454af4d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:00.168238 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50594250-760e-4fc2-ba7d-8c2454af4d5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:00.168238 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50594250-760e-4fc2-ba7d-8c2454af4d5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:00.168238 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50594250-760e-4fc2-ba7d-8c2454af4d5f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:00.176486 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 934/1877] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:00.181247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ff246-a7a1-4bce-80d6-2666629a0692 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:00.191668 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ff246-a7a1-4bce-80d6-2666629a0692 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:00.196726 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50594250-760e-4fc2-ba7d-8c2454af4d5f admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:00.197915 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93896d-7fa4-421e-be62-0e271af924ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:00.197915 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93896d-7fa4-421e-be62-0e271af924ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:00.197915 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93896d-7fa4-421e-be62-0e271af924ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:00.198203 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93896d-7fa4-421e-be62-0e271af924ce admin admin] PATH_INFO: `/v3/projects/ad0d4bc76c9c40b79877f323af3ef964/users/ee63a1d45a594925b7e59bc5eb353314/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:00.198783 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-50594250-760e-4fc2-ba7d-8c2454af4d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:00.213740 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50594250-760e-4fc2-ba7d-8c2454af4d5f admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:00.216560 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e93896d-7fa4-421e-be62-0e271af924ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad0d4bc76c9c40b79877f323af3ef964, user_id=ee63a1d45a594925b7e59bc5eb353314, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:00.219008 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795ebbe-6d27-4456-a227-361fb1b710f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:00.219380 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e93896d-7fa4-421e-be62-0e271af924ce admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:00.219643 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795ebbe-6d27-4456-a227-361fb1b710f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:00.219921 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795ebbe-6d27-4456-a227-361fb1b710f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:00.220230 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795ebbe-6d27-4456-a227-361fb1b710f6 admin admin] PATH_INFO: `/v3/projects/a316c02012534eca8a4ef12b1873e7e2` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:00.221354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f795ebbe-6d27-4456-a227-361fb1b710f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a316c02012534eca8a4ef12b1873e7e2)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:00.223236 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f795ebbe-6d27-4456-a227-361fb1b710f6 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:00.331739 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 944/1878] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:00 2022] PUT /identity/v3/projects/ad0d4bc76c9c40b79877f323af3ef964/users/ee63a1d45a594925b7e59bc5eb353314/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:00.342098 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3170f5c-fa3a-4bc6-adc4-5c45341665e0 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:00.342384 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3170f5c-fa3a-4bc6-adc4-5c45341665e0 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:00.342656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3170f5c-fa3a-4bc6-adc4-5c45341665e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:00.371257 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e3170f5c-fa3a-4bc6-adc4-5c45341665e0 None None] MFA Rules not processed for user `ee63a1d45a594925b7e59bc5eb353314`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:00.372541 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e3170f5c-fa3a-4bc6-adc4-5c45341665e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:00.380015 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f795ebbe-6d27-4456-a227-361fb1b710f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a316c02012534eca8a4ef12b1873e7e2', 'request_id': 'req-f795ebbe-6d27-4456-a227-361fb1b710f6'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:00.437995 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 945/1879] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:00 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:06:00.442581 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3170f5c-fa3a-4bc6-adc4-5c45341665e0 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:00.481571 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f795ebbe-6d27-4456-a227-361fb1b710f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a316c02012534eca8a4ef12b1873e7e2', 'request_id': 'req-f795ebbe-6d27-4456-a227-361fb1b710f6'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:00.484974 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:00.491570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f795ebbe-6d27-4456-a227-361fb1b710f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a316c02012534eca8a4ef12b1873e7e2', 'request_id': 'req-f795ebbe-6d27-4456-a227-361fb1b710f6'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:00.503475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f795ebbe-6d27-4456-a227-361fb1b710f6 admin admin] The token cache is being invalidate because project a316c02012534eca8a4ef12b1873e7e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:06:00.503891 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f795ebbe-6d27-4456-a227-361fb1b710f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:00.504618 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f795ebbe-6d27-4456-a227-361fb1b710f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:00.510699 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 935/1880] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:06:00 2022] DELETE /identity/v3/projects/a316c02012534eca8a4ef12b1873e7e2 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:00.512331 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054a62f-55ed-4bc6-a3e9-d0093d03bf4b None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:00.512692 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054a62f-55ed-4bc6-a3e9-d0093d03bf4b None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:00.513051 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054a62f-55ed-4bc6-a3e9-d0093d03bf4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:00.517758 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3170f5c-fa3a-4bc6-adc4-5c45341665e0 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:00.523005 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4786cc-87e5-41bc-8ad0-e89701c1e11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:00.523677 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4786cc-87e5-41bc-8ad0-e89701c1e11f admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:00.523958 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4786cc-87e5-41bc-8ad0-e89701c1e11f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:00.524264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4786cc-87e5-41bc-8ad0-e89701c1e11f admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:00.524915 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4786cc-87e5-41bc-8ad0-e89701c1e11f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:00.526833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4786cc-87e5-41bc-8ad0-e89701c1e11f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:00.536439 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-d054a62f-55ed-4bc6-a3e9-d0093d03bf4b None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:00.537777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d054a62f-55ed-4bc6-a3e9-d0093d03bf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:00.548644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 936/1881] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:00.556226 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89677fd0-09ae-41ad-b162-ffa7cacd4dc3 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:00.556518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89677fd0-09ae-41ad-b162-ffa7cacd4dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:00.556799 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89677fd0-09ae-41ad-b162-ffa7cacd4dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:00.577233 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-89677fd0-09ae-41ad-b162-ffa7cacd4dc3 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:00.578501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-89677fd0-09ae-41ad-b162-ffa7cacd4dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:00.587406 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 937/1882] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:06:00.591316 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89677fd0-09ae-41ad-b162-ffa7cacd4dc3 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:00.602939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89677fd0-09ae-41ad-b162-ffa7cacd4dc3 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:00.611871 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f672022-c8de-4288-a462-21bae925647b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:00.613176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f672022-c8de-4288-a462-21bae925647b admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:00.613176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f672022-c8de-4288-a462-21bae925647b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:00.613176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f672022-c8de-4288-a462-21bae925647b admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:00.614003 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 946/1883] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:06:00 2022] POST /identity/v3/projects => generated 401 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:06:00.616320 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f672022-c8de-4288-a462-21bae925647b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:06:00.616982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f672022-c8de-4288-a462-21bae925647b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:00.618378 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4786cc-87e5-41bc-8ad0-e89701c1e11f admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:00.620927 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bb4786cc-87e5-41bc-8ad0-e89701c1e11f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:00.621174 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f672022-c8de-4288-a462-21bae925647b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:00.637447 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 938/1884] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:06:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:06:00.645248 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f672022-c8de-4288-a462-21bae925647b admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:00.667993 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f672022-c8de-4288-a462-21bae925647b admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:00.708250 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4786cc-87e5-41bc-8ad0-e89701c1e11f admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:00.713403 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b1c73-a8e0-4a8b-8dc2-cb30814006e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:00.713817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b1c73-a8e0-4a8b-8dc2-cb30814006e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:00.714107 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b1c73-a8e0-4a8b-8dc2-cb30814006e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:00.714380 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b1c73-a8e0-4a8b-8dc2-cb30814006e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:00.715152 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b1c73-a8e0-4a8b-8dc2-cb30814006e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:00.717152 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b1c73-a8e0-4a8b-8dc2-cb30814006e4 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:00.740533 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42dcdd5-57ee-4f41-ada4-a89bd5737664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:00.741812 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42dcdd5-57ee-4f41-ada4-a89bd5737664 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:00.742158 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42dcdd5-57ee-4f41-ada4-a89bd5737664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:00.742503 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42dcdd5-57ee-4f41-ada4-a89bd5737664 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:00.743228 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42dcdd5-57ee-4f41-ada4-a89bd5737664 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:00.745315 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42dcdd5-57ee-4f41-ada4-a89bd5737664 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:00.800626 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 947/1885] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:06:00 2022] POST /identity/v3/users => generated 307 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:00.804626 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b1c73-a8e0-4a8b-8dc2-cb30814006e4 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:00.821429 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 939/1886] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:06:00 2022] POST /identity/v3/projects => generated 389 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:00.821768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b1c73-a8e0-4a8b-8dc2-cb30814006e4 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:00.825402 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42dcdd5-57ee-4f41-ada4-a89bd5737664 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:00.877944 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:00.879363 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:00.886727 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5138b1-39b9-44a8-8d70-150c40e72e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:00.887347 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5138b1-39b9-44a8-8d70-150c40e72e35 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:00.887630 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5138b1-39b9-44a8-8d70-150c40e72e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:00.887904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5138b1-39b9-44a8-8d70-150c40e72e35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:00.888585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5138b1-39b9-44a8-8d70-150c40e72e35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:00.890192 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5138b1-39b9-44a8-8d70-150c40e72e35 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:00.900049 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 948/1887] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:06:00.904139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5138b1-39b9-44a8-8d70-150c40e72e35 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:00.913767 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5138b1-39b9-44a8-8d70-150c40e72e35 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:00.917319 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42dcdd5-57ee-4f41-ada4-a89bd5737664 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:00.919036 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad36b1-2612-46d6-9a6d-003da8b23e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:00.919679 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad36b1-2612-46d6-9a6d-003da8b23e09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:00.923745 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad36b1-2612-46d6-9a6d-003da8b23e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:00.923745 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad36b1-2612-46d6-9a6d-003da8b23e09 admin admin] PATH_INFO: `/v3/projects/80cffe2a532f413580fec4bd842e4b09/users/f55fce9eeb0b4c7181a05b3b8601922b/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:00.923933 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8163a9-3716-45d4-8dfa-476141dd74e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:00.923933 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8163a9-3716-45d4-8dfa-476141dd74e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:00.923933 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8163a9-3716-45d4-8dfa-476141dd74e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:00.923933 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8163a9-3716-45d4-8dfa-476141dd74e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:00.924258 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8163a9-3716-45d4-8dfa-476141dd74e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:00.925777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8163a9-3716-45d4-8dfa-476141dd74e1 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:00.937182 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad36b1-2612-46d6-9a6d-003da8b23e09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80cffe2a532f413580fec4bd842e4b09, user_id=f55fce9eeb0b4c7181a05b3b8601922b, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:00.939973 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad36b1-2612-46d6-9a6d-003da8b23e09 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:00.999433 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 940/1888] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:06:00 2022] POST /identity/v3/users => generated 295 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:01.006513 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8163a9-3716-45d4-8dfa-476141dd74e1 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:01.016181 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8163a9-3716-45d4-8dfa-476141dd74e1 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:01.020851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b57ad41-14a7-422c-a5f8-c60290925b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:01.021437 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b57ad41-14a7-422c-a5f8-c60290925b0f admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:01.021722 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b57ad41-14a7-422c-a5f8-c60290925b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:01.024748 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b57ad41-14a7-422c-a5f8-c60290925b0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:01.024748 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b57ad41-14a7-422c-a5f8-c60290925b0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:01.024748 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b57ad41-14a7-422c-a5f8-c60290925b0f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:01.028821 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 949/1889] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:00 2022] PUT /identity/v3/projects/80cffe2a532f413580fec4bd842e4b09/users/f55fce9eeb0b4c7181a05b3b8601922b/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:06:01.032761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad36b1-2612-46d6-9a6d-003da8b23e09 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:01.034272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 941/1890] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:06:01.038563 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b57ad41-14a7-422c-a5f8-c60290925b0f admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:01.048713 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b57ad41-14a7-422c-a5f8-c60290925b0f admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:01.095293 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad36b1-2612-46d6-9a6d-003da8b23e09 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:01.100750 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b05dd-6717-42b0-88b4-c728a3e391fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:01.101359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b05dd-6717-42b0-88b4-c728a3e391fc admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:01.101636 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b05dd-6717-42b0-88b4-c728a3e391fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:01.101906 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b05dd-6717-42b0-88b4-c728a3e391fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:01.102925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b05dd-6717-42b0-88b4-c728a3e391fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:01.104584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b05dd-6717-42b0-88b4-c728a3e391fc admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:01.105599 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2434956c-9231-402d-bf2c-38d511a5f853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:01.106207 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2434956c-9231-402d-bf2c-38d511a5f853 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:01.106485 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2434956c-9231-402d-bf2c-38d511a5f853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:01.106793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2434956c-9231-402d-bf2c-38d511a5f853 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:01.107434 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2434956c-9231-402d-bf2c-38d511a5f853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:01.109034 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2434956c-9231-402d-bf2c-38d511a5f853 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:01.114926 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 950/1891] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:06:01.119381 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 942/1892] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:01.119719 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b05dd-6717-42b0-88b4-c728a3e391fc admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:01.123255 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2434956c-9231-402d-bf2c-38d511a5f853 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:01.129217 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b05dd-6717-42b0-88b4-c728a3e391fc admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:01.132780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2434956c-9231-402d-bf2c-38d511a5f853 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:01.133824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442acce8-cd5f-43b8-8a31-7bf426f9f9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:01.134462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442acce8-cd5f-43b8-8a31-7bf426f9f9f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:01.134744 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442acce8-cd5f-43b8-8a31-7bf426f9f9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:01.135011 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442acce8-cd5f-43b8-8a31-7bf426f9f9f1 admin admin] PATH_INFO: `/v3/projects/80cffe2a532f413580fec4bd842e4b09/users/f55fce9eeb0b4c7181a05b3b8601922b/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:01.136979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442acce8-cd5f-43b8-8a31-7bf426f9f9f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80cffe2a532f413580fec4bd842e4b09, user_id=f55fce9eeb0b4c7181a05b3b8601922b, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:01.138833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2554b6-2c7b-4435-ba9c-f97bb2bcdb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:01.139560 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442acce8-cd5f-43b8-8a31-7bf426f9f9f1 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:01.139779 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2554b6-2c7b-4435-ba9c-f97bb2bcdb46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:01.140062 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2554b6-2c7b-4435-ba9c-f97bb2bcdb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:01.140402 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2554b6-2c7b-4435-ba9c-f97bb2bcdb46 admin admin] PATH_INFO: `/v3/projects/60f3667625a8432cb99c584d7a87d737/users/790c350594894dfa94e7c413883dbc53/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:01.150886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 951/1893] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:01 2022] PUT /identity/v3/projects/80cffe2a532f413580fec4bd842e4b09/users/f55fce9eeb0b4c7181a05b3b8601922b/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:06:01.160123 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2554b6-2c7b-4435-ba9c-f97bb2bcdb46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60f3667625a8432cb99c584d7a87d737, user_id=790c350594894dfa94e7c413883dbc53, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:01.161966 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc32b0-a7eb-4884-ac19-d560f3c3db02 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:01.162467 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc32b0-a7eb-4884-ac19-d560f3c3db02 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:01.163114 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc32b0-a7eb-4884-ac19-d560f3c3db02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:01.163995 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2554b6-2c7b-4435-ba9c-f97bb2bcdb46 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:01.191252 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-21cc32b0-a7eb-4884-ac19-d560f3c3db02 None None] MFA Rules not processed for user `f55fce9eeb0b4c7181a05b3b8601922b`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:01.192480 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-21cc32b0-a7eb-4884-ac19-d560f3c3db02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:01.234169 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 943/1894] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:01 2022] PUT /identity/v3/projects/60f3667625a8432cb99c584d7a87d737/users/790c350594894dfa94e7c413883dbc53/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:06:01.245832 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171049d-2a07-4a4a-87b1-c2dc06b650f4 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:01.246123 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171049d-2a07-4a4a-87b1-c2dc06b650f4 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:01.246405 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171049d-2a07-4a4a-87b1-c2dc06b650f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:01.260562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 952/1895] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:01 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:06:01.271139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc32b0-a7eb-4884-ac19-d560f3c3db02 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:01.272591 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-21cc32b0-a7eb-4884-ac19-d560f3c3db02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:01.276906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-b171049d-2a07-4a4a-87b1-c2dc06b650f4 None None] MFA Rules not processed for user `790c350594894dfa94e7c413883dbc53`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:01.278125 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b171049d-2a07-4a4a-87b1-c2dc06b650f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:01.313340 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:01.330384 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc32b0-a7eb-4884-ac19-d560f3c3db02 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:01.347724 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 944/1896] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:01 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:06:01.352479 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171049d-2a07-4a4a-87b1-c2dc06b650f4 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:01.358113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:01.382229 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8d85c2-b4b6-466e-8156-e4dd9e20104d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:01.382854 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8d85c2-b4b6-466e-8156-e4dd9e20104d service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:01.383985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8d85c2-b4b6-466e-8156-e4dd9e20104d service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:01.383985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8d85c2-b4b6-466e-8156-e4dd9e20104d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:01.420650 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171049d-2a07-4a4a-87b1-c2dc06b650f4 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:01.425527 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c751d005-e3a3-4e6f-b37d-aae0b7799fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:01.426140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751d005-e3a3-4e6f-b37d-aae0b7799fb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:01.426429 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751d005-e3a3-4e6f-b37d-aae0b7799fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:01.426697 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751d005-e3a3-4e6f-b37d-aae0b7799fb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:01.427334 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c751d005-e3a3-4e6f-b37d-aae0b7799fb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:01.429349 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c751d005-e3a3-4e6f-b37d-aae0b7799fb9 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:01.447736 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8d85c2-b4b6-466e-8156-e4dd9e20104d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:01.449298 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8d85c2-b4b6-466e-8156-e4dd9e20104d service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:01.460497 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 953/1897] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:01 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:06:01.518587 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 945/1898] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:06:01 2022] POST /identity/v3/projects => generated 389 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:06:01.523264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8d85c2-b4b6-466e-8156-e4dd9e20104d service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:01.533773 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8d85c2-b4b6-466e-8156-e4dd9e20104d service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:01.602078 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed525323-64c2-4458-ac7f-600b49b6f03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:01.602699 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed525323-64c2-4458-ac7f-600b49b6f03d admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:01.602983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed525323-64c2-4458-ac7f-600b49b6f03d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:01.603295 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed525323-64c2-4458-ac7f-600b49b6f03d admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:01.603884 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed525323-64c2-4458-ac7f-600b49b6f03d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:01.605887 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed525323-64c2-4458-ac7f-600b49b6f03d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:01.741729 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c751d005-e3a3-4e6f-b37d-aae0b7799fb9 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:01.751099 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 954/1899] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:06:01 2022] POST /identity/v3/users => generated 301 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:06:01.756890 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed525323-64c2-4458-ac7f-600b49b6f03d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:01.768693 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed525323-64c2-4458-ac7f-600b49b6f03d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:01.775043 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca36b6c6-59f0-43b8-a5d3-e31b1fe1aee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:01.776223 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36b6c6-59f0-43b8-a5d3-e31b1fe1aee6 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:01.777338 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36b6c6-59f0-43b8-a5d3-e31b1fe1aee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:01.777818 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36b6c6-59f0-43b8-a5d3-e31b1fe1aee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:01.778959 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca36b6c6-59f0-43b8-a5d3-e31b1fe1aee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:01.781276 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca36b6c6-59f0-43b8-a5d3-e31b1fe1aee6 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:01.798656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 955/1900] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:06:01.800037 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca36b6c6-59f0-43b8-a5d3-e31b1fe1aee6 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:01.816424 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca36b6c6-59f0-43b8-a5d3-e31b1fe1aee6 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:01.818632 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c751d005-e3a3-4e6f-b37d-aae0b7799fb9 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:01.825028 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f679d3-7c87-4887-827a-6fddb9e440d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:01.826220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f679d3-7c87-4887-827a-6fddb9e440d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:01.826688 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f679d3-7c87-4887-827a-6fddb9e440d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:01.827142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f679d3-7c87-4887-827a-6fddb9e440d2 admin admin] PATH_INFO: `/v3/projects/53bcb219a4464b7aab765605e5be3b77/users/fd8f09d730da439cba9ad4f20e5808fd/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:01.827270 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a1195-7c1d-49c6-bf77-16c92b0e03c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:01.828877 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a1195-7c1d-49c6-bf77-16c92b0e03c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:01.829863 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a1195-7c1d-49c6-bf77-16c92b0e03c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:01.830145 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a1195-7c1d-49c6-bf77-16c92b0e03c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:01.832342 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-7f8a1195-7c1d-49c6-bf77-16c92b0e03c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:01.863389 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f679d3-7c87-4887-827a-6fddb9e440d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53bcb219a4464b7aab765605e5be3b77, user_id=fd8f09d730da439cba9ad4f20e5808fd, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:01.864423 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f679d3-7c87-4887-827a-6fddb9e440d2 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:01.868815 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:01.944293 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8a1195-7c1d-49c6-bf77-16c92b0e03c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:01.945017 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8a1195-7c1d-49c6-bf77-16c92b0e03c8 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:01.955836 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 946/1901] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:01 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:06:01.971626 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 956/1902] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:01 2022] PUT /identity/v3/projects/53bcb219a4464b7aab765605e5be3b77/users/fd8f09d730da439cba9ad4f20e5808fd/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:01.972972 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a1195-7c1d-49c6-bf77-16c92b0e03c8 service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:01.989418 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a1195-7c1d-49c6-bf77-16c92b0e03c8 service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:02.067499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:02.073533 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89ffa7-b958-45b8-a910-805f262c3559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:02.074346 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89ffa7-b958-45b8-a910-805f262c3559 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:02.074795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89ffa7-b958-45b8-a910-805f262c3559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:02.075244 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89ffa7-b958-45b8-a910-805f262c3559 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:02.076371 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da89ffa7-b958-45b8-a910-805f262c3559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:02.078161 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da89ffa7-b958-45b8-a910-805f262c3559 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:02.089059 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 947/1903] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:02.097151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f679d3-7c87-4887-827a-6fddb9e440d2 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:02.108472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f679d3-7c87-4887-827a-6fddb9e440d2 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:02.113185 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3d2f1-6a41-4f3d-a83a-76229964cad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:02.113963 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3d2f1-6a41-4f3d-a83a-76229964cad6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:02.114201 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3d2f1-6a41-4f3d-a83a-76229964cad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:02.114472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3d2f1-6a41-4f3d-a83a-76229964cad6 admin admin] PATH_INFO: `/v3/projects/53bcb219a4464b7aab765605e5be3b77/users/fd8f09d730da439cba9ad4f20e5808fd/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:02.117049 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d3d2f1-6a41-4f3d-a83a-76229964cad6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53bcb219a4464b7aab765605e5be3b77, user_id=fd8f09d730da439cba9ad4f20e5808fd, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:02.119768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d3d2f1-6a41-4f3d-a83a-76229964cad6 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:02.130396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 957/1904] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:02 2022] PUT /identity/v3/projects/53bcb219a4464b7aab765605e5be3b77/users/fd8f09d730da439cba9ad4f20e5808fd/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:02.141663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43349b5a-98e1-4ea7-8094-10b49a14f758 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:02.142020 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43349b5a-98e1-4ea7-8094-10b49a14f758 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:02.142406 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43349b5a-98e1-4ea7-8094-10b49a14f758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:02.170389 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-43349b5a-98e1-4ea7-8094-10b49a14f758 None None] MFA Rules not processed for user `fd8f09d730da439cba9ad4f20e5808fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:02.171656 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-43349b5a-98e1-4ea7-8094-10b49a14f758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:02.239166 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 948/1905] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:02 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:06:02.250889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3d2f1-6a41-4f3d-a83a-76229964cad6 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:02.252931 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-09d3d2f1-6a41-4f3d-a83a-76229964cad6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:02.314540 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3d2f1-6a41-4f3d-a83a-76229964cad6 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:02.318235 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a39f66-8b43-40cd-8d59-377ecb5e4902 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:02.319407 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a39f66-8b43-40cd-8d59-377ecb5e4902 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:02.319686 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a39f66-8b43-40cd-8d59-377ecb5e4902 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:02.319958 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a39f66-8b43-40cd-8d59-377ecb5e4902 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:02.329584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a39f66-8b43-40cd-8d59-377ecb5e4902 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:02.331014 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a39f66-8b43-40cd-8d59-377ecb5e4902 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:02.341596 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 958/1906] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:02 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:06:02.483332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43349b5a-98e1-4ea7-8094-10b49a14f758 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:02.557470 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43349b5a-98e1-4ea7-8094-10b49a14f758 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:02.562015 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9cf8a-0a36-4957-b398-424e0d9c13cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:02.562643 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9cf8a-0a36-4957-b398-424e0d9c13cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:02.562925 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9cf8a-0a36-4957-b398-424e0d9c13cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:02.563198 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9cf8a-0a36-4957-b398-424e0d9c13cd admin admin] PATH_INFO: `/v3/users/790c350594894dfa94e7c413883dbc53` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:02.564356 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f9cf8a-0a36-4957-b398-424e0d9c13cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=790c350594894dfa94e7c413883dbc53)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:02.566278 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f9cf8a-0a36-4957-b398-424e0d9c13cd admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:02.730741 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-60f9cf8a-0a36-4957-b398-424e0d9c13cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '790c350594894dfa94e7c413883dbc53'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:02.732125 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-60f9cf8a-0a36-4957-b398-424e0d9c13cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '790c350594894dfa94e7c413883dbc53'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:02.738838 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-60f9cf8a-0a36-4957-b398-424e0d9c13cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '790c350594894dfa94e7c413883dbc53'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:02.747884 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:02.752622 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-60f9cf8a-0a36-4957-b398-424e0d9c13cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '790c350594894dfa94e7c413883dbc53'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:02.753334 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-60f9cf8a-0a36-4957-b398-424e0d9c13cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '790c350594894dfa94e7c413883dbc53'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:02.863115 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 949/1907] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:06:02 2022] DELETE /identity/v3/users/790c350594894dfa94e7c413883dbc53 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:02.867750 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69d506-f3af-4fa3-bd5d-0bd1cc7a6886 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:02.868268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69d506-f3af-4fa3-bd5d-0bd1cc7a6886 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:02.868355 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69d506-f3af-4fa3-bd5d-0bd1cc7a6886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:02.888809 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ff69d506-f3af-4fa3-bd5d-0bd1cc7a6886 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:02.890042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ff69d506-f3af-4fa3-bd5d-0bd1cc7a6886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:02.981695 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 959/1908] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:03.006738 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9cf8a-0a36-4957-b398-424e0d9c13cd admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:03.019045 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-60f9cf8a-0a36-4957-b398-424e0d9c13cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:03.080159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9cf8a-0a36-4957-b398-424e0d9c13cd admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:03.083789 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ff586-2d8b-4a87-809d-0fea33475cfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:03.085013 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ff586-2d8b-4a87-809d-0fea33475cfb service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:03.085300 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ff586-2d8b-4a87-809d-0fea33475cfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:03.085572 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ff586-2d8b-4a87-809d-0fea33475cfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:03.095163 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ff586-2d8b-4a87-809d-0fea33475cfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:03.110024 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ff586-2d8b-4a87-809d-0fea33475cfb service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:03.121587 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 950/1909] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:06:03.649347 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69d506-f3af-4fa3-bd5d-0bd1cc7a6886 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:03.657609 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ff69d506-f3af-4fa3-bd5d-0bd1cc7a6886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:03.671978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69d506-f3af-4fa3-bd5d-0bd1cc7a6886 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:03.680258 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750597d6-24f1-4c83-b121-c7f178bf5a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:03.681001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750597d6-24f1-4c83-b121-c7f178bf5a70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:03.681283 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750597d6-24f1-4c83-b121-c7f178bf5a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:03.681578 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750597d6-24f1-4c83-b121-c7f178bf5a70 admin admin] PATH_INFO: `/v3/projects/60f3667625a8432cb99c584d7a87d737` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:03.682755 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750597d6-24f1-4c83-b121-c7f178bf5a70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60f3667625a8432cb99c584d7a87d737)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:03.684729 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750597d6-24f1-4c83-b121-c7f178bf5a70 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:03.817651 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-750597d6-24f1-4c83-b121-c7f178bf5a70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60f3667625a8432cb99c584d7a87d737', 'request_id': 'req-750597d6-24f1-4c83-b121-c7f178bf5a70'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:03.825021 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-750597d6-24f1-4c83-b121-c7f178bf5a70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60f3667625a8432cb99c584d7a87d737', 'request_id': 'req-750597d6-24f1-4c83-b121-c7f178bf5a70'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:03.932829 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-750597d6-24f1-4c83-b121-c7f178bf5a70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60f3667625a8432cb99c584d7a87d737', 'request_id': 'req-750597d6-24f1-4c83-b121-c7f178bf5a70'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:03.947219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-750597d6-24f1-4c83-b121-c7f178bf5a70 admin admin] The token cache is being invalidate because project 60f3667625a8432cb99c584d7a87d737 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:06:03.948281 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-750597d6-24f1-4c83-b121-c7f178bf5a70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:03.949118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-750597d6-24f1-4c83-b121-c7f178bf5a70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:03.951316 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 960/1910] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:06:03 2022] DELETE /identity/v3/projects/60f3667625a8432cb99c584d7a87d737 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:03.955364 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ff586-2d8b-4a87-809d-0fea33475cfb service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:03.966936 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ff586-2d8b-4a87-809d-0fea33475cfb service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:04.040045 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2290c1-8b51-4e1d-86f2-c4bf163a90af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:04.040794 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2290c1-8b51-4e1d-86f2-c4bf163a90af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:04.041135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2290c1-8b51-4e1d-86f2-c4bf163a90af admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:04.045455 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2290c1-8b51-4e1d-86f2-c4bf163a90af admin admin] PATH_INFO: `/v3/users/fd8f09d730da439cba9ad4f20e5808fd` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:04.045455 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2290c1-8b51-4e1d-86f2-c4bf163a90af admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd8f09d730da439cba9ad4f20e5808fd)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:04.045455 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2290c1-8b51-4e1d-86f2-c4bf163a90af admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:04.264830 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-cd2290c1-8b51-4e1d-86f2-c4bf163a90af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd8f09d730da439cba9ad4f20e5808fd'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:04.265477 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-cd2290c1-8b51-4e1d-86f2-c4bf163a90af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd8f09d730da439cba9ad4f20e5808fd'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:04.273002 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-cd2290c1-8b51-4e1d-86f2-c4bf163a90af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd8f09d730da439cba9ad4f20e5808fd'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:04.287655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-cd2290c1-8b51-4e1d-86f2-c4bf163a90af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd8f09d730da439cba9ad4f20e5808fd'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:04.287655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-cd2290c1-8b51-4e1d-86f2-c4bf163a90af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd8f09d730da439cba9ad4f20e5808fd'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:04.399694 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 951/1911] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:06:03 2022] DELETE /identity/v3/users/fd8f09d730da439cba9ad4f20e5808fd => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:04.855526 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750597d6-24f1-4c83-b121-c7f178bf5a70 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:04.857060 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-750597d6-24f1-4c83-b121-c7f178bf5a70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:04.877823 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750597d6-24f1-4c83-b121-c7f178bf5a70 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:04.884237 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:04.954940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c172d-5cbc-4ee8-ba29-a2be47218e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:04.955635 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c172d-5cbc-4ee8-ba29-a2be47218e1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:04.955858 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c172d-5cbc-4ee8-ba29-a2be47218e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:04.956158 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c172d-5cbc-4ee8-ba29-a2be47218e1e admin admin] PATH_INFO: `/v3/projects/53bcb219a4464b7aab765605e5be3b77` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:04.963136 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c172d-5cbc-4ee8-ba29-a2be47218e1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=53bcb219a4464b7aab765605e5be3b77)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:04.963136 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c172d-5cbc-4ee8-ba29-a2be47218e1e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:05.098531 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-745c172d-5cbc-4ee8-ba29-a2be47218e1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53bcb219a4464b7aab765605e5be3b77', 'request_id': 'req-745c172d-5cbc-4ee8-ba29-a2be47218e1e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:05.116311 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-745c172d-5cbc-4ee8-ba29-a2be47218e1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53bcb219a4464b7aab765605e5be3b77', 'request_id': 'req-745c172d-5cbc-4ee8-ba29-a2be47218e1e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:05.216464 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-745c172d-5cbc-4ee8-ba29-a2be47218e1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53bcb219a4464b7aab765605e5be3b77', 'request_id': 'req-745c172d-5cbc-4ee8-ba29-a2be47218e1e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:05.227797 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-745c172d-5cbc-4ee8-ba29-a2be47218e1e admin admin] The token cache is being invalidate because project 53bcb219a4464b7aab765605e5be3b77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:06:05.228111 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-745c172d-5cbc-4ee8-ba29-a2be47218e1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:05.228823 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-745c172d-5cbc-4ee8-ba29-a2be47218e1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:05.230317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 961/1912] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:06:04 2022] DELETE /identity/v3/projects/53bcb219a4464b7aab765605e5be3b77 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:05.235583 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4d5b2-85fb-47c9-bbeb-82fc17a49b2e None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:05.235890 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4d5b2-85fb-47c9-bbeb-82fc17a49b2e None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:05.236190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4d5b2-85fb-47c9-bbeb-82fc17a49b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:05.251592 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:05.258044 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-fdf4d5b2-85fb-47c9-bbeb-82fc17a49b2e None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:05.259349 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-fdf4d5b2-85fb-47c9-bbeb-82fc17a49b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:05.322999 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 952/1913] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:05.332170 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc80cb9-8fee-4be5-b07e-5b0b3771e8f8 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:05.332552 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc80cb9-8fee-4be5-b07e-5b0b3771e8f8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:05.332893 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc80cb9-8fee-4be5-b07e-5b0b3771e8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:05.357964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-adc80cb9-8fee-4be5-b07e-5b0b3771e8f8 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:05.357964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-adc80cb9-8fee-4be5-b07e-5b0b3771e8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:05.363682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 962/1914] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:05.368177 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf4d5b2-85fb-47c9-bbeb-82fc17a49b2e None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:05.377942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf4d5b2-85fb-47c9-bbeb-82fc17a49b2e None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:05.386681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960fcf5-e06f-4a16-922f-63b73dcd5c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:05.386681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960fcf5-e06f-4a16-922f-63b73dcd5c32 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:05.386681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960fcf5-e06f-4a16-922f-63b73dcd5c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:05.386681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960fcf5-e06f-4a16-922f-63b73dcd5c32 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:05.387044 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960fcf5-e06f-4a16-922f-63b73dcd5c32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:06:05.387044 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960fcf5-e06f-4a16-922f-63b73dcd5c32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:05.387044 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960fcf5-e06f-4a16-922f-63b73dcd5c32 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:05.400027 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 953/1915] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:06:05.404523 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc80cb9-8fee-4be5-b07e-5b0b3771e8f8 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:05.415926 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc80cb9-8fee-4be5-b07e-5b0b3771e8f8 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:05.421020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17b013-eec7-4986-8067-d6a80034106c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:05.421758 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17b013-eec7-4986-8067-d6a80034106c admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:05.422147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17b013-eec7-4986-8067-d6a80034106c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:05.422571 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17b013-eec7-4986-8067-d6a80034106c admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:05.423337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca17b013-eec7-4986-8067-d6a80034106c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:05.425540 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca17b013-eec7-4986-8067-d6a80034106c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:05.549331 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 963/1916] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:06:05 2022] POST /identity/v3/projects => generated 381 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:05.553659 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960fcf5-e06f-4a16-922f-63b73dcd5c32 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:05.563771 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960fcf5-e06f-4a16-922f-63b73dcd5c32 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:05.621253 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f7cea-2add-4508-8a4a-2b8a02545317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:05.621793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f7cea-2add-4508-8a4a-2b8a02545317 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:05.622071 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f7cea-2add-4508-8a4a-2b8a02545317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:05.622342 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f7cea-2add-4508-8a4a-2b8a02545317 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:05.622974 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f7cea-2add-4508-8a4a-2b8a02545317 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:05.624936 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f7cea-2add-4508-8a4a-2b8a02545317 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:05.714297 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 954/1917] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:06:05 2022] POST /identity/v3/users => generated 291 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:05.718492 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17b013-eec7-4986-8067-d6a80034106c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:05.729022 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17b013-eec7-4986-8067-d6a80034106c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:05.733869 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76cf80-8c5d-453c-bc79-4b701e3420a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:05.734465 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76cf80-8c5d-453c-bc79-4b701e3420a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:05.734749 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76cf80-8c5d-453c-bc79-4b701e3420a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:05.735018 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76cf80-8c5d-453c-bc79-4b701e3420a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:05.735661 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba76cf80-8c5d-453c-bc79-4b701e3420a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:05.737313 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba76cf80-8c5d-453c-bc79-4b701e3420a4 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:05.747308 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 964/1918] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:05.751237 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f7cea-2add-4508-8a4a-2b8a02545317 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:05.761537 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f7cea-2add-4508-8a4a-2b8a02545317 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:05.766042 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698f9d1-0703-4b84-8633-42db3efc91ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:05.766624 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698f9d1-0703-4b84-8633-42db3efc91ca admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:05.766899 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698f9d1-0703-4b84-8633-42db3efc91ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:05.767169 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698f9d1-0703-4b84-8633-42db3efc91ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:05.767806 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9698f9d1-0703-4b84-8633-42db3efc91ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:05.769423 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9698f9d1-0703-4b84-8633-42db3efc91ca admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:05.786952 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 955/1919] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:05.789719 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76cf80-8c5d-453c-bc79-4b701e3420a4 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:05.800031 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76cf80-8c5d-453c-bc79-4b701e3420a4 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:05.804754 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f72c5-2d19-46f3-8219-c88e6e174479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:05.805471 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f72c5-2d19-46f3-8219-c88e6e174479 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:05.805759 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f72c5-2d19-46f3-8219-c88e6e174479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:05.806035 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f72c5-2d19-46f3-8219-c88e6e174479 admin admin] PATH_INFO: `/v3/projects/dcf5e84b8a0a43e08e2203a80008180b/users/73e92ec39ca44c76b7ca59cc64decd8f/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:05.822715 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f72c5-2d19-46f3-8219-c88e6e174479 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcf5e84b8a0a43e08e2203a80008180b, user_id=73e92ec39ca44c76b7ca59cc64decd8f, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:05.826387 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f72c5-2d19-46f3-8219-c88e6e174479 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:05.945338 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 965/1920] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:05 2022] PUT /identity/v3/projects/dcf5e84b8a0a43e08e2203a80008180b/users/73e92ec39ca44c76b7ca59cc64decd8f/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:05.959798 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b8c8e-adf8-4168-bba2-2eb3be0511b2 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:05.959798 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b8c8e-adf8-4168-bba2-2eb3be0511b2 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:05.959798 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b8c8e-adf8-4168-bba2-2eb3be0511b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:05.991521 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-919b8c8e-adf8-4168-bba2-2eb3be0511b2 None None] MFA Rules not processed for user `73e92ec39ca44c76b7ca59cc64decd8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:05.993047 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-919b8c8e-adf8-4168-bba2-2eb3be0511b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:06.060990 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 956/1921] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:06.065491 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f72c5-2d19-46f3-8219-c88e6e174479 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:06.125852 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f72c5-2d19-46f3-8219-c88e6e174479 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:06.130521 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa071072-af3d-4e57-810f-1155aa9105d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:06.131064 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa071072-af3d-4e57-810f-1155aa9105d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:06.131910 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa071072-af3d-4e57-810f-1155aa9105d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:06.132213 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa071072-af3d-4e57-810f-1155aa9105d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:06.132841 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa071072-af3d-4e57-810f-1155aa9105d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:06.134746 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa071072-af3d-4e57-810f-1155aa9105d7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:06.251602 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 966/1922] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:06:06 2022] POST /identity/v3/projects => generated 379 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:06.257080 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b8c8e-adf8-4168-bba2-2eb3be0511b2 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:06.271255 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b8c8e-adf8-4168-bba2-2eb3be0511b2 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:06.342910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81358c2-ff38-4cae-a6e1-78b31b13d601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:06.343576 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81358c2-ff38-4cae-a6e1-78b31b13d601 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:06.343911 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81358c2-ff38-4cae-a6e1-78b31b13d601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:06.344248 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81358c2-ff38-4cae-a6e1-78b31b13d601 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:06.344892 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81358c2-ff38-4cae-a6e1-78b31b13d601 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:06.346804 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81358c2-ff38-4cae-a6e1-78b31b13d601 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:06.451529 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 957/1923] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:06:06 2022] POST /identity/v3/users => generated 296 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:06.455749 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa071072-af3d-4e57-810f-1155aa9105d7 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:06.466046 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa071072-af3d-4e57-810f-1155aa9105d7 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:06.470612 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96297fe-1e33-45d5-a2d8-198160659828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:06.471194 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96297fe-1e33-45d5-a2d8-198160659828 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:06.471475 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96297fe-1e33-45d5-a2d8-198160659828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:06.471753 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96297fe-1e33-45d5-a2d8-198160659828 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:06.472604 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96297fe-1e33-45d5-a2d8-198160659828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:06.474258 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96297fe-1e33-45d5-a2d8-198160659828 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:06.486160 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 967/1924] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:06.491444 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81358c2-ff38-4cae-a6e1-78b31b13d601 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:06.502594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81358c2-ff38-4cae-a6e1-78b31b13d601 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:06.507411 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d5fed1-81b9-452d-a840-eea8b812ff47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:06.512381 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5fed1-81b9-452d-a840-eea8b812ff47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:06.512381 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5fed1-81b9-452d-a840-eea8b812ff47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:06.512381 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5fed1-81b9-452d-a840-eea8b812ff47 admin admin] PATH_INFO: `/v3/projects/9fab208f7a2e4626a8fd89a24d12b8b4/users/9689e4d2d7a741bf926b737d31651fb9/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:06.524150 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d5fed1-81b9-452d-a840-eea8b812ff47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fab208f7a2e4626a8fd89a24d12b8b4, user_id=9689e4d2d7a741bf926b737d31651fb9, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:06.526856 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d5fed1-81b9-452d-a840-eea8b812ff47 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:06.599734 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 958/1925] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:06 2022] PUT /identity/v3/projects/9fab208f7a2e4626a8fd89a24d12b8b4/users/9689e4d2d7a741bf926b737d31651fb9/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:06.604598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96297fe-1e33-45d5-a2d8-198160659828 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:06.615048 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96297fe-1e33-45d5-a2d8-198160659828 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:06.669543 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c3302d-c745-4397-a26f-eaae01b5195e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:06.670131 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3302d-c745-4397-a26f-eaae01b5195e admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:06.670407 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3302d-c745-4397-a26f-eaae01b5195e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:06.670678 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3302d-c745-4397-a26f-eaae01b5195e admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:06.671308 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c3302d-c745-4397-a26f-eaae01b5195e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:06.672988 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c3302d-c745-4397-a26f-eaae01b5195e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:06.684376 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 968/1926] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:06.688598 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d5fed1-81b9-452d-a840-eea8b812ff47 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:06.699449 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d5fed1-81b9-452d-a840-eea8b812ff47 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:06.704855 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bbbf42-0e3f-416a-9948-1eb6e43fdebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:06.705520 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbbf42-0e3f-416a-9948-1eb6e43fdebd admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:06.705896 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbbf42-0e3f-416a-9948-1eb6e43fdebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:06.706235 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbbf42-0e3f-416a-9948-1eb6e43fdebd admin admin] PATH_INFO: `/v3/projects/9fab208f7a2e4626a8fd89a24d12b8b4/users/9689e4d2d7a741bf926b737d31651fb9/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:06.708250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bbbf42-0e3f-416a-9948-1eb6e43fdebd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fab208f7a2e4626a8fd89a24d12b8b4, user_id=9689e4d2d7a741bf926b737d31651fb9, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:06.711511 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bbbf42-0e3f-416a-9948-1eb6e43fdebd admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:06.721961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 959/1927] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:06 2022] PUT /identity/v3/projects/9fab208f7a2e4626a8fd89a24d12b8b4/users/9689e4d2d7a741bf926b737d31651fb9/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:06.733036 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bb5e5-b5e3-42d8-a5b9-9b31ded32af2 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:06.733807 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bb5e5-b5e3-42d8-a5b9-9b31ded32af2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:06.733807 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bb5e5-b5e3-42d8-a5b9-9b31ded32af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:06.763240 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-2a2bb5e5-b5e3-42d8-a5b9-9b31ded32af2 None None] MFA Rules not processed for user `9689e4d2d7a741bf926b737d31651fb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:06.764505 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2a2bb5e5-b5e3-42d8-a5b9-9b31ded32af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:06.845206 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 969/1928] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:06 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:06.856778 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bbbf42-0e3f-416a-9948-1eb6e43fdebd admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:06.861860 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-97bbbf42-0e3f-416a-9948-1eb6e43fdebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:06.911094 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2bb5e5-b5e3-42d8-a5b9-9b31ded32af2 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:06.913288 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2a2bb5e5-b5e3-42d8-a5b9-9b31ded32af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:06.931565 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bbbf42-0e3f-416a-9948-1eb6e43fdebd admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:06.935488 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137f524-289e-4e42-b3dc-cb316398b143 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:06.936309 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137f524-289e-4e42-b3dc-cb316398b143 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:06.936731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137f524-289e-4e42-b3dc-cb316398b143 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:06.937082 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137f524-289e-4e42-b3dc-cb316398b143 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:06.947322 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2137f524-289e-4e42-b3dc-cb316398b143 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:06.949081 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2137f524-289e-4e42-b3dc-cb316398b143 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:06.959962 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 960/1929] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:06 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:06:06.992502 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2bb5e5-b5e3-42d8-a5b9-9b31ded32af2 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:06.997076 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803cd442-7831-40b3-8f38-91b532a6ed9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:06.997708 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cd442-7831-40b3-8f38-91b532a6ed9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:06.997983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cd442-7831-40b3-8f38-91b532a6ed9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:06.998254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cd442-7831-40b3-8f38-91b532a6ed9b admin admin] PATH_INFO: `/v3/users/ad32d78ac4f9435a8ef0e96689be950f` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:06.999412 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803cd442-7831-40b3-8f38-91b532a6ed9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad32d78ac4f9435a8ef0e96689be950f)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:07.001915 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803cd442-7831-40b3-8f38-91b532a6ed9b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:07.316604 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:07.342524 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-803cd442-7831-40b3-8f38-91b532a6ed9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad32d78ac4f9435a8ef0e96689be950f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:07.343261 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-803cd442-7831-40b3-8f38-91b532a6ed9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad32d78ac4f9435a8ef0e96689be950f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:07.344139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-803cd442-7831-40b3-8f38-91b532a6ed9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad32d78ac4f9435a8ef0e96689be950f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:07.358850 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-803cd442-7831-40b3-8f38-91b532a6ed9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad32d78ac4f9435a8ef0e96689be950f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:07.376656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-803cd442-7831-40b3-8f38-91b532a6ed9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad32d78ac4f9435a8ef0e96689be950f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:07.475686 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 970/1930] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:06:06 2022] DELETE /identity/v3/users/ad32d78ac4f9435a8ef0e96689be950f => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:07.486827 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e849858-fbf3-4da7-b9d4-7bb46ea2f921 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:07.486827 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e849858-fbf3-4da7-b9d4-7bb46ea2f921 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:07.486827 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e849858-fbf3-4da7-b9d4-7bb46ea2f921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:07.508800 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-4e849858-fbf3-4da7-b9d4-7bb46ea2f921 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:07.510017 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-4e849858-fbf3-4da7-b9d4-7bb46ea2f921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:07.571250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 961/1931] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:07.581571 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803cd442-7831-40b3-8f38-91b532a6ed9b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:07.583022 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-803cd442-7831-40b3-8f38-91b532a6ed9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:07.646372 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e849858-fbf3-4da7-b9d4-7bb46ea2f921 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:07.648223 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-4e849858-fbf3-4da7-b9d4-7bb46ea2f921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:07.648649 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803cd442-7831-40b3-8f38-91b532a6ed9b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:07.652503 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232695f6-008f-4607-810b-b1371946d9c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:07.653197 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232695f6-008f-4607-810b-b1371946d9c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:07.653479 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232695f6-008f-4607-810b-b1371946d9c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:07.653760 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232695f6-008f-4607-810b-b1371946d9c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:07.661543 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e849858-fbf3-4da7-b9d4-7bb46ea2f921 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:07.665686 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232695f6-008f-4607-810b-b1371946d9c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:07.667243 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232695f6-008f-4607-810b-b1371946d9c6 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:07.672981 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50195c80-6ad9-4a60-8525-64343672c327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:07.673695 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50195c80-6ad9-4a60-8525-64343672c327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:07.673982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50195c80-6ad9-4a60-8525-64343672c327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:07.674387 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50195c80-6ad9-4a60-8525-64343672c327 admin admin] PATH_INFO: `/v3/users/ee63a1d45a594925b7e59bc5eb353314` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:07.675640 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50195c80-6ad9-4a60-8525-64343672c327 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee63a1d45a594925b7e59bc5eb353314)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:07.677463 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 971/1932] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:06:07.677734 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50195c80-6ad9-4a60-8525-64343672c327 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:07.687457 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232695f6-008f-4607-810b-b1371946d9c6 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:07.695824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-232695f6-008f-4607-810b-b1371946d9c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:07.699951 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:07.779169 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232695f6-008f-4607-810b-b1371946d9c6 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:07.783627 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6e437-f1a7-4c3c-a1e0-0af201bc4c65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:07.784273 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6e437-f1a7-4c3c-a1e0-0af201bc4c65 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:07.784557 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6e437-f1a7-4c3c-a1e0-0af201bc4c65 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:07.784833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6e437-f1a7-4c3c-a1e0-0af201bc4c65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:07.786929 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bdb6e437-f1a7-4c3c-a1e0-0af201bc4c65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:07.854636 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6e437-f1a7-4c3c-a1e0-0af201bc4c65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:07.856173 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6e437-f1a7-4c3c-a1e0-0af201bc4c65 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:07.872376 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 972/1933] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:06:08.104553 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6e437-f1a7-4c3c-a1e0-0af201bc4c65 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:08.106912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bdb6e437-f1a7-4c3c-a1e0-0af201bc4c65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:08.124837 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6e437-f1a7-4c3c-a1e0-0af201bc4c65 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:08.131389 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d9e14-e05d-488d-bc8e-7f15cfedeee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:08.132175 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d9e14-e05d-488d-bc8e-7f15cfedeee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:08.132537 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d9e14-e05d-488d-bc8e-7f15cfedeee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:08.132892 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d9e14-e05d-488d-bc8e-7f15cfedeee3 admin admin] PATH_INFO: `/v3/users/73e92ec39ca44c76b7ca59cc64decd8f` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:08.134784 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d9e14-e05d-488d-bc8e-7f15cfedeee3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73e92ec39ca44c76b7ca59cc64decd8f)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:08.137395 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d9e14-e05d-488d-bc8e-7f15cfedeee3 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:08.152001 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-50195c80-6ad9-4a60-8525-64343672c327 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee63a1d45a594925b7e59bc5eb353314'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:08.152809 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-50195c80-6ad9-4a60-8525-64343672c327 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee63a1d45a594925b7e59bc5eb353314'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:08.160964 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-50195c80-6ad9-4a60-8525-64343672c327 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee63a1d45a594925b7e59bc5eb353314'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:08.193596 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-50195c80-6ad9-4a60-8525-64343672c327 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee63a1d45a594925b7e59bc5eb353314'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:08.194617 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-50195c80-6ad9-4a60-8525-64343672c327 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee63a1d45a594925b7e59bc5eb353314'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:08.274563 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 962/1934] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:06:07 2022] DELETE /identity/v3/users/ee63a1d45a594925b7e59bc5eb353314 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:08.277346 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb74d86-cac7-45ac-acce-e369fda5f5c2 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:08.277738 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb74d86-cac7-45ac-acce-e369fda5f5c2 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:08.278105 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb74d86-cac7-45ac-acce-e369fda5f5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:08.308328 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-dcb74d86-cac7-45ac-acce-e369fda5f5c2 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:08.309904 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-dcb74d86-cac7-45ac-acce-e369fda5f5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:08.372695 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 963/1935] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:06:08.382131 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb74d86-cac7-45ac-acce-e369fda5f5c2 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:08.383442 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-dcb74d86-cac7-45ac-acce-e369fda5f5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:08.423577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2f8d9e14-e05d-488d-bc8e-7f15cfedeee3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73e92ec39ca44c76b7ca59cc64decd8f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:08.423577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2f8d9e14-e05d-488d-bc8e-7f15cfedeee3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73e92ec39ca44c76b7ca59cc64decd8f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:08.423577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2f8d9e14-e05d-488d-bc8e-7f15cfedeee3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73e92ec39ca44c76b7ca59cc64decd8f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:08.430808 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2f8d9e14-e05d-488d-bc8e-7f15cfedeee3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73e92ec39ca44c76b7ca59cc64decd8f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:08.448915 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2f8d9e14-e05d-488d-bc8e-7f15cfedeee3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73e92ec39ca44c76b7ca59cc64decd8f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:08.454641 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb74d86-cac7-45ac-acce-e369fda5f5c2 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:08.458647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb9eac-066d-4601-8cb1-d0d4e770cf27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:08.459846 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb9eac-066d-4601-8cb1-d0d4e770cf27 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:08.460162 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb9eac-066d-4601-8cb1-d0d4e770cf27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:08.460444 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb9eac-066d-4601-8cb1-d0d4e770cf27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:08.462230 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-61eb9eac-066d-4601-8cb1-d0d4e770cf27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:08.475448 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb9eac-066d-4601-8cb1-d0d4e770cf27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:08.476943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb9eac-066d-4601-8cb1-d0d4e770cf27 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:08.486943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 964/1936] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:06:08.487844 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb9eac-066d-4601-8cb1-d0d4e770cf27 service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:08.489137 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-61eb9eac-066d-4601-8cb1-d0d4e770cf27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:08.501391 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb9eac-066d-4601-8cb1-d0d4e770cf27 service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:08.506933 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9f6b8-a158-425a-b11d-30f37b2787fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:08.507709 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9f6b8-a158-425a-b11d-30f37b2787fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:08.515167 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9f6b8-a158-425a-b11d-30f37b2787fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:08.515167 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9f6b8-a158-425a-b11d-30f37b2787fc admin admin] PATH_INFO: `/v3/projects/fca4e790f8d14e3e8038e402f2ba2959` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:08.515167 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d9f6b8-a158-425a-b11d-30f37b2787fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=fca4e790f8d14e3e8038e402f2ba2959)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:08.515167 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d9f6b8-a158-425a-b11d-30f37b2787fc admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:08.550858 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 973/1937] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:06:08 2022] DELETE /identity/v3/users/73e92ec39ca44c76b7ca59cc64decd8f => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:08.555221 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfdcd2-dd7c-40b8-a17e-efc602d2cb1b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:08.555506 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfdcd2-dd7c-40b8-a17e-efc602d2cb1b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:08.555779 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfdcd2-dd7c-40b8-a17e-efc602d2cb1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:08.576888 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-30cfdcd2-dd7c-40b8-a17e-efc602d2cb1b None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:08.578273 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-30cfdcd2-dd7c-40b8-a17e-efc602d2cb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:08.636506 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 974/1938] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:08.645972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cfdcd2-dd7c-40b8-a17e-efc602d2cb1b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:08.647912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-30cfdcd2-dd7c-40b8-a17e-efc602d2cb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:08.673886 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-13d9f6b8-a158-425a-b11d-30f37b2787fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fca4e790f8d14e3e8038e402f2ba2959', 'request_id': 'req-13d9f6b8-a158-425a-b11d-30f37b2787fc'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:08.708342 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cfdcd2-dd7c-40b8-a17e-efc602d2cb1b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:08.754797 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-13d9f6b8-a158-425a-b11d-30f37b2787fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fca4e790f8d14e3e8038e402f2ba2959', 'request_id': 'req-13d9f6b8-a158-425a-b11d-30f37b2787fc'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:08.763692 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-13d9f6b8-a158-425a-b11d-30f37b2787fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fca4e790f8d14e3e8038e402f2ba2959', 'request_id': 'req-13d9f6b8-a158-425a-b11d-30f37b2787fc'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:08.763855 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1168e082-8690-43f0-bed7-e763c7467316 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:08.764426 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168e082-8690-43f0-bed7-e763c7467316 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:08.764707 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168e082-8690-43f0-bed7-e763c7467316 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:08.764981 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168e082-8690-43f0-bed7-e763c7467316 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:08.777552 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-13d9f6b8-a158-425a-b11d-30f37b2787fc admin admin] The token cache is being invalidate because project fca4e790f8d14e3e8038e402f2ba2959 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:06:08.778165 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-13d9f6b8-a158-425a-b11d-30f37b2787fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:08.779053 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-13d9f6b8-a158-425a-b11d-30f37b2787fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:08.783941 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 965/1939] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:06:08 2022] DELETE /identity/v3/projects/fca4e790f8d14e3e8038e402f2ba2959 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:08.785216 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9f6b8-a158-425a-b11d-30f37b2787fc admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:08.786877 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-13d9f6b8-a158-425a-b11d-30f37b2787fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:08.840743 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1168e082-8690-43f0-bed7-e763c7467316 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:08.842200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1168e082-8690-43f0-bed7-e763c7467316 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:08.852494 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 975/1940] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:06:08.861306 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9f6b8-a158-425a-b11d-30f37b2787fc admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:08.866857 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f38c235-54df-4f3b-85a1-3bbd685c26ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:08.867491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38c235-54df-4f3b-85a1-3bbd685c26ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:08.867771 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38c235-54df-4f3b-85a1-3bbd685c26ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:08.868190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38c235-54df-4f3b-85a1-3bbd685c26ac admin admin] PATH_INFO: `/v3/users/c715ae48efc440258c7014c7c52de669` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:08.869375 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f38c235-54df-4f3b-85a1-3bbd685c26ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=c715ae48efc440258c7014c7c52de669)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:08.871301 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f38c235-54df-4f3b-85a1-3bbd685c26ac admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:08.983805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1168e082-8690-43f0-bed7-e763c7467316 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:08.985932 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1168e082-8690-43f0-bed7-e763c7467316 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:09.003151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1168e082-8690-43f0-bed7-e763c7467316 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:09.017025 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04bd5e-7bef-4784-92ce-3b661246835f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:09.018025 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04bd5e-7bef-4784-92ce-3b661246835f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:09.020471 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04bd5e-7bef-4784-92ce-3b661246835f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:09.021020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04bd5e-7bef-4784-92ce-3b661246835f admin admin] PATH_INFO: `/v3/projects/ad0d4bc76c9c40b79877f323af3ef964` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:09.022554 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c04bd5e-7bef-4784-92ce-3b661246835f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad0d4bc76c9c40b79877f323af3ef964)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:09.025284 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c04bd5e-7bef-4784-92ce-3b661246835f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:09.159749 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8f38c235-54df-4f3b-85a1-3bbd685c26ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c715ae48efc440258c7014c7c52de669'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:09.160440 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8f38c235-54df-4f3b-85a1-3bbd685c26ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c715ae48efc440258c7014c7c52de669'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:09.176805 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8f38c235-54df-4f3b-85a1-3bbd685c26ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c715ae48efc440258c7014c7c52de669'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:09.195367 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8f38c235-54df-4f3b-85a1-3bbd685c26ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c715ae48efc440258c7014c7c52de669'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:09.196321 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8f38c235-54df-4f3b-85a1-3bbd685c26ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c715ae48efc440258c7014c7c52de669'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:09.215319 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5c04bd5e-7bef-4784-92ce-3b661246835f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad0d4bc76c9c40b79877f323af3ef964', 'request_id': 'req-5c04bd5e-7bef-4784-92ce-3b661246835f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:09.231649 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5c04bd5e-7bef-4784-92ce-3b661246835f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad0d4bc76c9c40b79877f323af3ef964', 'request_id': 'req-5c04bd5e-7bef-4784-92ce-3b661246835f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:09.276286 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:09.340720 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 966/1941] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:06:08 2022] DELETE /identity/v3/users/c715ae48efc440258c7014c7c52de669 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:09.494413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5c04bd5e-7bef-4784-92ce-3b661246835f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad0d4bc76c9c40b79877f323af3ef964', 'request_id': 'req-5c04bd5e-7bef-4784-92ce-3b661246835f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:09.503531 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f38c235-54df-4f3b-85a1-3bbd685c26ac admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:09.507726 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8f38c235-54df-4f3b-85a1-3bbd685c26ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:09.535291 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5c04bd5e-7bef-4784-92ce-3b661246835f admin admin] The token cache is being invalidate because project ad0d4bc76c9c40b79877f323af3ef964 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:06:09.539382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5c04bd5e-7bef-4784-92ce-3b661246835f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:09.539382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5c04bd5e-7bef-4784-92ce-3b661246835f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:09.541507 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 976/1942] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:06:08 2022] DELETE /identity/v3/projects/ad0d4bc76c9c40b79877f323af3ef964 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:09.545581 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04bd5e-7bef-4784-92ce-3b661246835f admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:09.551705 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5c04bd5e-7bef-4784-92ce-3b661246835f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:09.590671 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:09.655461 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f38c235-54df-4f3b-85a1-3bbd685c26ac admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:09.660931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58119e6c-4e59-4f59-a92c-01ddd69b6ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:09.661722 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58119e6c-4e59-4f59-a92c-01ddd69b6ef5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:09.662011 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58119e6c-4e59-4f59-a92c-01ddd69b6ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:09.662881 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58119e6c-4e59-4f59-a92c-01ddd69b6ef5 admin admin] PATH_INFO: `/v3/projects/dcf5e84b8a0a43e08e2203a80008180b` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:09.664055 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58119e6c-4e59-4f59-a92c-01ddd69b6ef5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcf5e84b8a0a43e08e2203a80008180b)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:09.666241 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58119e6c-4e59-4f59-a92c-01ddd69b6ef5 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:09.683486 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04bd5e-7bef-4784-92ce-3b661246835f admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:09.690020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5e11d-a6bc-403b-9a92-4471195edced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:09.690768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5e11d-a6bc-403b-9a92-4471195edced admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:09.691376 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5e11d-a6bc-403b-9a92-4471195edced admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:09.691723 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5e11d-a6bc-403b-9a92-4471195edced admin admin] PATH_INFO: `/v3/users/f55fce9eeb0b4c7181a05b3b8601922b` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:09.693340 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5e11d-a6bc-403b-9a92-4471195edced admin admin] RBAC: Authorizing `identity:delete_user(user_id=f55fce9eeb0b4c7181a05b3b8601922b)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:09.695550 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5e11d-a6bc-403b-9a92-4471195edced admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:09.815752 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-58119e6c-4e59-4f59-a92c-01ddd69b6ef5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcf5e84b8a0a43e08e2203a80008180b', 'request_id': 'req-58119e6c-4e59-4f59-a92c-01ddd69b6ef5'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:09.910707 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-58119e6c-4e59-4f59-a92c-01ddd69b6ef5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcf5e84b8a0a43e08e2203a80008180b', 'request_id': 'req-58119e6c-4e59-4f59-a92c-01ddd69b6ef5'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:09.919602 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-58119e6c-4e59-4f59-a92c-01ddd69b6ef5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcf5e84b8a0a43e08e2203a80008180b', 'request_id': 'req-58119e6c-4e59-4f59-a92c-01ddd69b6ef5'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:09.932847 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-58119e6c-4e59-4f59-a92c-01ddd69b6ef5 admin admin] The token cache is being invalidate because project dcf5e84b8a0a43e08e2203a80008180b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:06:09.933148 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-58119e6c-4e59-4f59-a92c-01ddd69b6ef5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:09.934361 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-58119e6c-4e59-4f59-a92c-01ddd69b6ef5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:09.937341 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 967/1943] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:06:09 2022] DELETE /identity/v3/projects/dcf5e84b8a0a43e08e2203a80008180b => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:09.938725 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1c84b-4b0a-49b4-b78b-604189ffd285 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:09.938975 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1c84b-4b0a-49b4-b78b-604189ffd285 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:09.939257 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1c84b-4b0a-49b4-b78b-604189ffd285 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:09.942366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 968/1944] 10.222.0.48 () {54 vars in 921 bytes} [Fri Sep 9 21:06:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:06:09.943412 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed1c84b-4b0a-49b4-b78b-604189ffd285 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:09.945650 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2ed1c84b-4b0a-49b4-b78b-604189ffd285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:09.950243 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-26e5e11d-a6bc-403b-9a92-4471195edced admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f55fce9eeb0b4c7181a05b3b8601922b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:09.951220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-26e5e11d-a6bc-403b-9a92-4471195edced admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f55fce9eeb0b4c7181a05b3b8601922b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:09.954508 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-26e5e11d-a6bc-403b-9a92-4471195edced admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f55fce9eeb0b4c7181a05b3b8601922b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:09.965965 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-26e5e11d-a6bc-403b-9a92-4471195edced admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f55fce9eeb0b4c7181a05b3b8601922b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:09.990456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-26e5e11d-a6bc-403b-9a92-4471195edced admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f55fce9eeb0b4c7181a05b3b8601922b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:10.066405 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed1c84b-4b0a-49b4-b78b-604189ffd285 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:10.071065 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e4e0e-de8a-42e5-a621-cd61855d365f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:10.072618 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e4e0e-de8a-42e5-a621-cd61855d365f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:10.072903 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e4e0e-de8a-42e5-a621-cd61855d365f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:10.073181 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e4e0e-de8a-42e5-a621-cd61855d365f admin admin] PATH_INFO: `/v3/users/9689e4d2d7a741bf926b737d31651fb9` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:10.074367 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e4e0e-de8a-42e5-a621-cd61855d365f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9689e4d2d7a741bf926b737d31651fb9)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:10.076689 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e4e0e-de8a-42e5-a621-cd61855d365f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:10.093739 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 977/1945] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:06:09 2022] DELETE /identity/v3/users/f55fce9eeb0b4c7181a05b3b8601922b => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:10.095324 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bef3a6-3c93-4410-be67-9ad64ee0617b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:10.096204 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bef3a6-3c93-4410-be67-9ad64ee0617b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:10.096568 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bef3a6-3c93-4410-be67-9ad64ee0617b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:10.123259 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-20bef3a6-3c93-4410-be67-9ad64ee0617b None None] MFA Rules not processed for user `613b845a9b4d432daf8d478fc3970708`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:10.124535 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-20bef3a6-3c93-4410-be67-9ad64ee0617b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:10.211979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 978/1946] 10.222.0.48 () {58 vars in 1018 bytes} [Fri Sep 9 21:06:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:10.213210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bef3a6-3c93-4410-be67-9ad64ee0617b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:10.214798 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-20bef3a6-3c93-4410-be67-9ad64ee0617b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:10.330410 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bef3a6-3c93-4410-be67-9ad64ee0617b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:10.338403 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e642c-3302-4f86-bdba-41629259aac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:10.339259 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e642c-3302-4f86-bdba-41629259aac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:10.340457 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e642c-3302-4f86-bdba-41629259aac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:10.340743 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e642c-3302-4f86-bdba-41629259aac5 admin admin] PATH_INFO: `/v3/projects/0d6aeb92851c44a69579a436efacddda` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:10.343746 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e642c-3302-4f86-bdba-41629259aac5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d6aeb92851c44a69579a436efacddda)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:10.344198 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e642c-3302-4f86-bdba-41629259aac5 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:10.345235 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b54e4e0e-de8a-42e5-a621-cd61855d365f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9689e4d2d7a741bf926b737d31651fb9'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:10.346028 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b54e4e0e-de8a-42e5-a621-cd61855d365f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9689e4d2d7a741bf926b737d31651fb9'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:10.359952 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b54e4e0e-de8a-42e5-a621-cd61855d365f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9689e4d2d7a741bf926b737d31651fb9'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:10.377469 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b54e4e0e-de8a-42e5-a621-cd61855d365f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9689e4d2d7a741bf926b737d31651fb9'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:10.378391 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b54e4e0e-de8a-42e5-a621-cd61855d365f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9689e4d2d7a741bf926b737d31651fb9'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:10.451330 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 969/1947] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:06:09 2022] DELETE /identity/v3/users/9689e4d2d7a741bf926b737d31651fb9 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:10.452423 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e4e0e-de8a-42e5-a621-cd61855d365f admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:10.453920 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b54e4e0e-de8a-42e5-a621-cd61855d365f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:10.522841 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-896e642c-3302-4f86-bdba-41629259aac5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d6aeb92851c44a69579a436efacddda', 'request_id': 'req-896e642c-3302-4f86-bdba-41629259aac5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:10.530016 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-896e642c-3302-4f86-bdba-41629259aac5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d6aeb92851c44a69579a436efacddda', 'request_id': 'req-896e642c-3302-4f86-bdba-41629259aac5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:10.551472 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e4e0e-de8a-42e5-a621-cd61855d365f admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:10.632960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6f0f6-d73a-419f-922a-b63809df2708 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:10.634481 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6f0f6-d73a-419f-922a-b63809df2708 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:10.634481 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6f0f6-d73a-419f-922a-b63809df2708 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:10.634691 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6f0f6-d73a-419f-922a-b63809df2708 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:10.637014 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-97f6f0f6-d73a-419f-922a-b63809df2708 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:10.641585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-896e642c-3302-4f86-bdba-41629259aac5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d6aeb92851c44a69579a436efacddda', 'request_id': 'req-896e642c-3302-4f86-bdba-41629259aac5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:10.652879 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6f0f6-d73a-419f-922a-b63809df2708 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:10.654725 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6f0f6-d73a-419f-922a-b63809df2708 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:10.658230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-896e642c-3302-4f86-bdba-41629259aac5 admin admin] The token cache is being invalidate because project 0d6aeb92851c44a69579a436efacddda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:06:10.658533 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-896e642c-3302-4f86-bdba-41629259aac5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:10.664630 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-896e642c-3302-4f86-bdba-41629259aac5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:10.666196 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 979/1948] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:06:10 2022] DELETE /identity/v3/projects/0d6aeb92851c44a69579a436efacddda => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:10.671061 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01125862-b43e-419b-81af-d32e147834f5 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:10.671354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01125862-b43e-419b-81af-d32e147834f5 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:10.671640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01125862-b43e-419b-81af-d32e147834f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:10.679381 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 970/1949] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:06:10.714121 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-01125862-b43e-419b-81af-d32e147834f5 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:10.715359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-01125862-b43e-419b-81af-d32e147834f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:10.727554 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:10.841551 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 980/1950] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:06:10.843643 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c1192-2994-4855-b0b7-ca44ec3a1ac6 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:10.844296 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c1192-2994-4855-b0b7-ca44ec3a1ac6 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:10.844575 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c1192-2994-4855-b0b7-ca44ec3a1ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:10.870874 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-8c6c1192-2994-4855-b0b7-ca44ec3a1ac6 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:10.872443 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8c6c1192-2994-4855-b0b7-ca44ec3a1ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:10.881146 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 971/1951] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:10.885539 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01125862-b43e-419b-81af-d32e147834f5 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:10.895517 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01125862-b43e-419b-81af-d32e147834f5 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:10.900534 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26214b0-cba6-4f6a-bd49-123249591042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:10.901146 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26214b0-cba6-4f6a-bd49-123249591042 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:10.901423 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26214b0-cba6-4f6a-bd49-123249591042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:10.901717 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26214b0-cba6-4f6a-bd49-123249591042 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:10.902701 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26214b0-cba6-4f6a-bd49-123249591042 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:06:10.902701 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26214b0-cba6-4f6a-bd49-123249591042 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:10.904415 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26214b0-cba6-4f6a-bd49-123249591042 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:10.922777 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 981/1952] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:06:10.923593 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c1192-2994-4855-b0b7-ca44ec3a1ac6 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:10.925326 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26214b0-cba6-4f6a-bd49-123249591042 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:10.926175 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8c6c1192-2994-4855-b0b7-ca44ec3a1ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:10.943175 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26214b0-cba6-4f6a-bd49-123249591042 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:10.946078 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c1192-2994-4855-b0b7-ca44ec3a1ac6 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:10.948176 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de56a6c-51a9-4b64-9529-d5bfb9a0ab17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:10.948789 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de56a6c-51a9-4b64-9529-d5bfb9a0ab17 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:10.949260 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de56a6c-51a9-4b64-9529-d5bfb9a0ab17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:10.949527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de56a6c-51a9-4b64-9529-d5bfb9a0ab17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:10.950159 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de56a6c-51a9-4b64-9529-d5bfb9a0ab17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:10.952786 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de56a6c-51a9-4b64-9529-d5bfb9a0ab17 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:10.963525 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a977efaa-2b84-45eb-9924-44f84a5766e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:10.964733 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a977efaa-2b84-45eb-9924-44f84a5766e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:10.966814 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a977efaa-2b84-45eb-9924-44f84a5766e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:10.968324 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a977efaa-2b84-45eb-9924-44f84a5766e9 admin admin] PATH_INFO: `/v3/projects/80cffe2a532f413580fec4bd842e4b09` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:10.971323 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a977efaa-2b84-45eb-9924-44f84a5766e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80cffe2a532f413580fec4bd842e4b09)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:10.976604 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a977efaa-2b84-45eb-9924-44f84a5766e9 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:11.146515 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 982/1953] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:06:10 2022] POST /identity/v3/projects => generated 365 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:11.152457 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de56a6c-51a9-4b64-9529-d5bfb9a0ab17 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:11.216103 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a977efaa-2b84-45eb-9924-44f84a5766e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80cffe2a532f413580fec4bd842e4b09', 'request_id': 'req-a977efaa-2b84-45eb-9924-44f84a5766e9'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:11.249177 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de56a6c-51a9-4b64-9529-d5bfb9a0ab17 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:11.277755 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:11.307563 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a977efaa-2b84-45eb-9924-44f84a5766e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80cffe2a532f413580fec4bd842e4b09', 'request_id': 'req-a977efaa-2b84-45eb-9924-44f84a5766e9'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:11.320675 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a977efaa-2b84-45eb-9924-44f84a5766e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80cffe2a532f413580fec4bd842e4b09', 'request_id': 'req-a977efaa-2b84-45eb-9924-44f84a5766e9'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:11.353435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a977efaa-2b84-45eb-9924-44f84a5766e9 admin admin] The token cache is being invalidate because project 80cffe2a532f413580fec4bd842e4b09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:06:11.354318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a977efaa-2b84-45eb-9924-44f84a5766e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:11.355565 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a977efaa-2b84-45eb-9924-44f84a5766e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:11.364178 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 972/1954] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:06:10 2022] DELETE /identity/v3/projects/80cffe2a532f413580fec4bd842e4b09 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:06:11.365705 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4a5c5-74b3-42d7-b102-1bcf06d2e606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:11.366349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4a5c5-74b3-42d7-b102-1bcf06d2e606 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:11.366628 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4a5c5-74b3-42d7-b102-1bcf06d2e606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:11.371287 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4a5c5-74b3-42d7-b102-1bcf06d2e606 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:11.371287 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4a5c5-74b3-42d7-b102-1bcf06d2e606 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:11.371474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a25df-c9a3-4e55-a9b3-f913a66ee685 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:11.371474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a25df-c9a3-4e55-a9b3-f913a66ee685 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:11.371474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a25df-c9a3-4e55-a9b3-f913a66ee685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:11.372928 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4a5c5-74b3-42d7-b102-1bcf06d2e606 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:11.411476 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-0e0a25df-c9a3-4e55-a9b3-f913a66ee685 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:11.413534 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0e0a25df-c9a3-4e55-a9b3-f913a66ee685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:11.424655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 973/1955] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:11.433793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015aba64-26a8-41ca-bd2b-57f8dce3698d None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:11.434078 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015aba64-26a8-41ca-bd2b-57f8dce3698d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:11.434343 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015aba64-26a8-41ca-bd2b-57f8dce3698d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:11.478127 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 983/1956] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:06:11 2022] POST /identity/v3/users => generated 283 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:06:11.480479 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-015aba64-26a8-41ca-bd2b-57f8dce3698d None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:11.480653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-015aba64-26a8-41ca-bd2b-57f8dce3698d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:11.482997 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4a5c5-74b3-42d7-b102-1bcf06d2e606 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:11.485781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-84c4a5c5-74b3-42d7-b102-1bcf06d2e606 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:11.505063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 974/1957] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:11.506751 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015aba64-26a8-41ca-bd2b-57f8dce3698d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:11.517861 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4a5c5-74b3-42d7-b102-1bcf06d2e606 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:11.526054 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015aba64-26a8-41ca-bd2b-57f8dce3698d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:11.533355 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac5bc4-da28-4b18-a6f6-097f7fc3afb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:11.534720 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac5bc4-da28-4b18-a6f6-097f7fc3afb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:11.535291 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac5bc4-da28-4b18-a6f6-097f7fc3afb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:11.535761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac5bc4-da28-4b18-a6f6-097f7fc3afb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:11.536888 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f93d6-0572-47c3-9aa8-b238f741263a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:11.543385 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac5bc4-da28-4b18-a6f6-097f7fc3afb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:11.543512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f93d6-0572-47c3-9aa8-b238f741263a admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:11.543512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f93d6-0572-47c3-9aa8-b238f741263a admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:11.543512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f93d6-0572-47c3-9aa8-b238f741263a admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:11.543512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f93d6-0572-47c3-9aa8-b238f741263a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:06:11.543512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f93d6-0572-47c3-9aa8-b238f741263a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:11.543512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f93d6-0572-47c3-9aa8-b238f741263a admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:11.543913 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac5bc4-da28-4b18-a6f6-097f7fc3afb2 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:11.554187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 975/1958] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:06:11.558986 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f93d6-0572-47c3-9aa8-b238f741263a admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:11.577433 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f93d6-0572-47c3-9aa8-b238f741263a admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:11.577778 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 984/1959] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:11.580015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac5bc4-da28-4b18-a6f6-097f7fc3afb2 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:11.595681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5891fe-f0e8-4fd3-8beb-3b9e348e0af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:11.596469 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5891fe-f0e8-4fd3-8beb-3b9e348e0af6 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:11.600313 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5891fe-f0e8-4fd3-8beb-3b9e348e0af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:11.600313 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5891fe-f0e8-4fd3-8beb-3b9e348e0af6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:11.600313 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5891fe-f0e8-4fd3-8beb-3b9e348e0af6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:11.600313 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5891fe-f0e8-4fd3-8beb-3b9e348e0af6 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:11.603107 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac5bc4-da28-4b18-a6f6-097f7fc3afb2 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:11.617142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2542a-a38c-4ed0-853b-09d93007c974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:11.617859 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2542a-a38c-4ed0-853b-09d93007c974 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:11.618139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2542a-a38c-4ed0-853b-09d93007c974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:11.618455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2542a-a38c-4ed0-853b-09d93007c974 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:11.619121 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f2542a-a38c-4ed0-853b-09d93007c974 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:11.623464 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f2542a-a38c-4ed0-853b-09d93007c974 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:11.649338 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 985/1960] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:11.652031 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2542a-a38c-4ed0-853b-09d93007c974 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:11.665524 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2542a-a38c-4ed0-853b-09d93007c974 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:11.671199 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f6e9d-5a77-4ec2-81ba-40f39a1999f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:11.671909 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f6e9d-5a77-4ec2-81ba-40f39a1999f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:11.672228 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f6e9d-5a77-4ec2-81ba-40f39a1999f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:11.672511 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f6e9d-5a77-4ec2-81ba-40f39a1999f7 admin admin] PATH_INFO: `/v3/projects/951eca8992d347be9672b9f3a27741af/users/9f56bf91efd54c74a9b4a358f2f9417e/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:11.690164 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f6e9d-5a77-4ec2-81ba-40f39a1999f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=951eca8992d347be9672b9f3a27741af, user_id=9f56bf91efd54c74a9b4a358f2f9417e, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:11.693322 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f6e9d-5a77-4ec2-81ba-40f39a1999f7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:11.735351 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 976/1961] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:06:11 2022] POST /identity/v3/projects => generated 383 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:06:11.751131 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5891fe-f0e8-4fd3-8beb-3b9e348e0af6 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:11.789728 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 986/1962] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:11 2022] PUT /identity/v3/projects/951eca8992d347be9672b9f3a27741af/users/9f56bf91efd54c74a9b4a358f2f9417e/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:11.791428 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f6e9d-5a77-4ec2-81ba-40f39a1999f7 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:11.793487 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ef1f6e9d-5a77-4ec2-81ba-40f39a1999f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:11.931824 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5891fe-f0e8-4fd3-8beb-3b9e348e0af6 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:11.953931 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f6e9d-5a77-4ec2-81ba-40f39a1999f7 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:11.959861 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c21963-4e25-4d46-a346-7c1a675e5f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:11.961004 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c21963-4e25-4d46-a346-7c1a675e5f17 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:11.962408 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c21963-4e25-4d46-a346-7c1a675e5f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:11.964768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c21963-4e25-4d46-a346-7c1a675e5f17 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:11.969786 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c21963-4e25-4d46-a346-7c1a675e5f17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:11.973604 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c21963-4e25-4d46-a346-7c1a675e5f17 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:11.978917 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f45166-76d1-4e8e-836f-df52e2123a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:11.980001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f45166-76d1-4e8e-836f-df52e2123a64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:11.980517 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f45166-76d1-4e8e-836f-df52e2123a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:11.980999 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f45166-76d1-4e8e-836f-df52e2123a64 admin admin] PATH_INFO: `/v3/projects/9fab208f7a2e4626a8fd89a24d12b8b4` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:11.984622 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f45166-76d1-4e8e-836f-df52e2123a64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fab208f7a2e4626a8fd89a24d12b8b4)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:11.988721 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f45166-76d1-4e8e-836f-df52e2123a64 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:12.116570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 977/1963] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:06:11 2022] POST /identity/v3/users => generated 292 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:12.123236 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77dd026-8558-42c7-b10f-38d9317607ba None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:12.124113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77dd026-8558-42c7-b10f-38d9317607ba None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:12.124745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77dd026-8558-42c7-b10f-38d9317607ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:12.169908 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d7f45166-76d1-4e8e-836f-df52e2123a64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fab208f7a2e4626a8fd89a24d12b8b4', 'request_id': 'req-d7f45166-76d1-4e8e-836f-df52e2123a64'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:12.179378 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d7f45166-76d1-4e8e-836f-df52e2123a64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fab208f7a2e4626a8fd89a24d12b8b4', 'request_id': 'req-d7f45166-76d1-4e8e-836f-df52e2123a64'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:12.205915 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-c77dd026-8558-42c7-b10f-38d9317607ba None None] MFA Rules not processed for user `9f56bf91efd54c74a9b4a358f2f9417e`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:12.208502 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c77dd026-8558-42c7-b10f-38d9317607ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:12.278262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d7f45166-76d1-4e8e-836f-df52e2123a64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fab208f7a2e4626a8fd89a24d12b8b4', 'request_id': 'req-d7f45166-76d1-4e8e-836f-df52e2123a64'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:12.297773 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d7f45166-76d1-4e8e-836f-df52e2123a64 admin admin] The token cache is being invalidate because project 9fab208f7a2e4626a8fd89a24d12b8b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:06:12.298800 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d7f45166-76d1-4e8e-836f-df52e2123a64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:12.299720 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d7f45166-76d1-4e8e-836f-df52e2123a64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:12.309680 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 978/1965] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:12.309680 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77dd026-8558-42c7-b10f-38d9317607ba None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:12.311158 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 987/1965] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:06:11 2022] DELETE /identity/v3/projects/9fab208f7a2e4626a8fd89a24d12b8b4 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:12.313659 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f356c-41e5-4438-aeb3-527607ea5014 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:12.315084 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f356c-41e5-4438-aeb3-527607ea5014 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:12.315675 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f356c-41e5-4438-aeb3-527607ea5014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:12.342697 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ff7f356c-41e5-4438-aeb3-527607ea5014 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:12.345023 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ff7f356c-41e5-4438-aeb3-527607ea5014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:12.467007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77dd026-8558-42c7-b10f-38d9317607ba None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:12.473113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ce114-b033-444c-b260-1afa3b459e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:12.473875 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ce114-b033-444c-b260-1afa3b459e5e admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:12.475225 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ce114-b033-444c-b260-1afa3b459e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:12.475605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ce114-b033-444c-b260-1afa3b459e5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:12.477264 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3ce114-b033-444c-b260-1afa3b459e5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:12.479890 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3ce114-b033-444c-b260-1afa3b459e5e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:12.496378 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 979/1966] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:12.496894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ce114-b033-444c-b260-1afa3b459e5e admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:12.497293 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 988/1967] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:12.498910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-6b3ce114-b033-444c-b260-1afa3b459e5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:12.499213 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddc7cb-a142-409b-9981-3589b91a3620 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:12.499657 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddc7cb-a142-409b-9981-3589b91a3620 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:12.500099 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddc7cb-a142-409b-9981-3589b91a3620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:12.516130 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ce114-b033-444c-b260-1afa3b459e5e admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:12.522699 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb575038-e22e-452b-865e-25fa2a450264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:12.523086 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb575038-e22e-452b-865e-25fa2a450264 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:12.523414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb575038-e22e-452b-865e-25fa2a450264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:12.523791 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb575038-e22e-452b-865e-25fa2a450264 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:12.524596 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb575038-e22e-452b-865e-25fa2a450264 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:12.526651 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb575038-e22e-452b-865e-25fa2a450264 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:12.542461 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c8ddc7cb-a142-409b-9981-3589b91a3620 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:12.544152 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c8ddc7cb-a142-409b-9981-3589b91a3620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:12.555406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 989/1968] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:12.556307 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddc7cb-a142-409b-9981-3589b91a3620 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:12.557700 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c8ddc7cb-a142-409b-9981-3589b91a3620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:12.569788 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddc7cb-a142-409b-9981-3589b91a3620 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:12.574404 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e551238-8543-4ad0-a8bd-723e7c008fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:12.575004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e551238-8543-4ad0-a8bd-723e7c008fec admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:12.578163 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e551238-8543-4ad0-a8bd-723e7c008fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:12.578163 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e551238-8543-4ad0-a8bd-723e7c008fec admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:12.578163 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e551238-8543-4ad0-a8bd-723e7c008fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:12.578163 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e551238-8543-4ad0-a8bd-723e7c008fec admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:12.591865 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 990/1969] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:12.591865 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e551238-8543-4ad0-a8bd-723e7c008fec admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:12.606532 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e551238-8543-4ad0-a8bd-723e7c008fec admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:12.612925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1268a7-32ed-461a-90d6-fa4b336a509b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:12.613747 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1268a7-32ed-461a-90d6-fa4b336a509b admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:12.614481 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1268a7-32ed-461a-90d6-fa4b336a509b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:12.614481 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1268a7-32ed-461a-90d6-fa4b336a509b admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:12.615368 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1268a7-32ed-461a-90d6-fa4b336a509b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:06:12.615786 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1268a7-32ed-461a-90d6-fa4b336a509b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:12.617534 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1268a7-32ed-461a-90d6-fa4b336a509b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:12.634024 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 991/1970] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:06:12.634968 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1268a7-32ed-461a-90d6-fa4b336a509b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:12.647211 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1268a7-32ed-461a-90d6-fa4b336a509b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:12.654866 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220ccbd-65fa-48cd-a552-c76c44fc3c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:12.655595 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220ccbd-65fa-48cd-a552-c76c44fc3c16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:12.655913 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220ccbd-65fa-48cd-a552-c76c44fc3c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:12.656179 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220ccbd-65fa-48cd-a552-c76c44fc3c16 admin admin] PATH_INFO: `/v3/projects/e56b485033634c68a287c68f41c93a9a/users/a5ecdcb3f7844718aea399b360e9e32f/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:12.663609 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 980/1971] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:06:12 2022] POST /identity/v3/projects => generated 365 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:12.665805 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb575038-e22e-452b-865e-25fa2a450264 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:12.674850 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5220ccbd-65fa-48cd-a552-c76c44fc3c16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e56b485033634c68a287c68f41c93a9a, user_id=a5ecdcb3f7844718aea399b360e9e32f, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:12.678421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5220ccbd-65fa-48cd-a552-c76c44fc3c16 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:12.737345 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb575038-e22e-452b-865e-25fa2a450264 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:12.743781 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9373a3c1-b051-4c12-b329-2bd617734ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:12.744548 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373a3c1-b051-4c12-b329-2bd617734ecb admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:12.744924 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373a3c1-b051-4c12-b329-2bd617734ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:12.745318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373a3c1-b051-4c12-b329-2bd617734ecb admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:12.746059 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9373a3c1-b051-4c12-b329-2bd617734ecb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:12.748129 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9373a3c1-b051-4c12-b329-2bd617734ecb admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:12.785161 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 992/1972] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:12 2022] PUT /identity/v3/projects/e56b485033634c68a287c68f41c93a9a/users/a5ecdcb3f7844718aea399b360e9e32f/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:12.789740 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220ccbd-65fa-48cd-a552-c76c44fc3c16 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:12.831289 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:12.871212 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 981/1973] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:06:12 2022] POST /identity/v3/projects => generated 383 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:12.872775 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc856e1-985c-4fb9-a848-03e60c0733df None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:12.873230 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc856e1-985c-4fb9-a848-03e60c0733df None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:12.873800 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc856e1-985c-4fb9-a848-03e60c0733df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:12.910638 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-bcc856e1-985c-4fb9-a848-03e60c0733df None None] MFA Rules not processed for user `a5ecdcb3f7844718aea399b360e9e32f`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:12.912832 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-bcc856e1-985c-4fb9-a848-03e60c0733df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:12.914719 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220ccbd-65fa-48cd-a552-c76c44fc3c16 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:13.012398 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32bbd2-ed2e-4367-83f3-23cf2a15891b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:13.013756 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32bbd2-ed2e-4367-83f3-23cf2a15891b admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:13.014102 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32bbd2-ed2e-4367-83f3-23cf2a15891b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:13.015283 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32bbd2-ed2e-4367-83f3-23cf2a15891b admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:13.015403 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f32bbd2-ed2e-4367-83f3-23cf2a15891b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:13.018006 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f32bbd2-ed2e-4367-83f3-23cf2a15891b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:13.025086 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 982/1974] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:13.026830 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc856e1-985c-4fb9-a848-03e60c0733df None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:13.037030 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc856e1-985c-4fb9-a848-03e60c0733df None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:13.041786 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e576d-7a23-43f9-99a4-da3eeb60eace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:13.042394 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e576d-7a23-43f9-99a4-da3eeb60eace admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:13.042674 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e576d-7a23-43f9-99a4-da3eeb60eace admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:13.042949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e576d-7a23-43f9-99a4-da3eeb60eace admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:13.043610 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e576d-7a23-43f9-99a4-da3eeb60eace admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:13.046733 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e576d-7a23-43f9-99a4-da3eeb60eace admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:13.126957 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 993/1975] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:06:12 2022] POST /identity/v3/users => generated 289 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:13.127794 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32bbd2-ed2e-4367-83f3-23cf2a15891b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:13.129343 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0f32bbd2-ed2e-4367-83f3-23cf2a15891b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:13.138254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:13.141605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 983/1976] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:06:13 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:13.141605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e576d-7a23-43f9-99a4-da3eeb60eace admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:13.145804 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32bbd2-ed2e-4367-83f3-23cf2a15891b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:13.149856 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e576d-7a23-43f9-99a4-da3eeb60eace admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:13.151321 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd74f67-f9fa-48d6-887d-958de660164b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:13.152018 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd74f67-f9fa-48d6-887d-958de660164b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:13.152359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd74f67-f9fa-48d6-887d-958de660164b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:13.152634 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd74f67-f9fa-48d6-887d-958de660164b admin admin] PATH_INFO: `/v3/users/35fa3f1d4f7241a0b55281bfa3c31f60` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:13.153800 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd74f67-f9fa-48d6-887d-958de660164b admin admin] RBAC: Authorizing `identity:delete_user(user_id=35fa3f1d4f7241a0b55281bfa3c31f60)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:13.154737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de38309-31d8-472f-9a50-9e7cd39cc011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:13.155261 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de38309-31d8-472f-9a50-9e7cd39cc011 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:13.155543 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de38309-31d8-472f-9a50-9e7cd39cc011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:13.155810 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd74f67-f9fa-48d6-887d-958de660164b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:13.155900 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de38309-31d8-472f-9a50-9e7cd39cc011 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:13.156498 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de38309-31d8-472f-9a50-9e7cd39cc011 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:13.158446 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de38309-31d8-472f-9a50-9e7cd39cc011 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:13.242351 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 984/1977] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:06:13 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:13.242351 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de38309-31d8-472f-9a50-9e7cd39cc011 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:13.318288 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de38309-31d8-472f-9a50-9e7cd39cc011 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:13.323699 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbefa0-5e1e-4249-b12c-3ad3c4b69737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:13.324433 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbefa0-5e1e-4249-b12c-3ad3c4b69737 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:13.324717 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbefa0-5e1e-4249-b12c-3ad3c4b69737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:13.325025 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbefa0-5e1e-4249-b12c-3ad3c4b69737 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:13.325634 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebbefa0-5e1e-4249-b12c-3ad3c4b69737 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:13.327479 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebbefa0-5e1e-4249-b12c-3ad3c4b69737 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:13.339360 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 985/1978] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:13.343196 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbefa0-5e1e-4249-b12c-3ad3c4b69737 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:13.354615 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbefa0-5e1e-4249-b12c-3ad3c4b69737 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:13.359641 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e05a6-ead3-46b2-9a66-bd2b482db6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:13.360341 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e05a6-ead3-46b2-9a66-bd2b482db6c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:13.360618 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e05a6-ead3-46b2-9a66-bd2b482db6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:13.360893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e05a6-ead3-46b2-9a66-bd2b482db6c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:13.361535 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3e05a6-ead3-46b2-9a66-bd2b482db6c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:13.363197 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3e05a6-ead3-46b2-9a66-bd2b482db6c6 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:13.374974 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 986/1979] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:13.378778 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e05a6-ead3-46b2-9a66-bd2b482db6c6 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:13.389883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6bd74f67-f9fa-48d6-887d-958de660164b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35fa3f1d4f7241a0b55281bfa3c31f60'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:13.390532 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e05a6-ead3-46b2-9a66-bd2b482db6c6 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:13.390657 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6bd74f67-f9fa-48d6-887d-958de660164b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35fa3f1d4f7241a0b55281bfa3c31f60'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:13.391515 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6bd74f67-f9fa-48d6-887d-958de660164b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35fa3f1d4f7241a0b55281bfa3c31f60'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:13.402068 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24440aa-62bf-44d0-914d-7830adb9f120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:13.402068 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24440aa-62bf-44d0-914d-7830adb9f120 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:13.402068 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24440aa-62bf-44d0-914d-7830adb9f120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:13.402068 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24440aa-62bf-44d0-914d-7830adb9f120 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:13.402417 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24440aa-62bf-44d0-914d-7830adb9f120 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:13.402505 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6bd74f67-f9fa-48d6-887d-958de660164b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35fa3f1d4f7241a0b55281bfa3c31f60'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:13.405318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24440aa-62bf-44d0-914d-7830adb9f120 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:13.423888 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6bd74f67-f9fa-48d6-887d-958de660164b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35fa3f1d4f7241a0b55281bfa3c31f60'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:13.578163 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 987/1980] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:06:13 2022] POST /identity/v3/users => generated 298 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:13.578843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24440aa-62bf-44d0-914d-7830adb9f120 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:13.583180 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 994/1981] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:06:13 2022] DELETE /identity/v3/users/35fa3f1d4f7241a0b55281bfa3c31f60 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:13.583180 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd74f67-f9fa-48d6-887d-958de660164b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:13.588964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6bd74f67-f9fa-48d6-887d-958de660164b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:13.590928 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d24440aa-62bf-44d0-914d-7830adb9f120 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:13.620732 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24440aa-62bf-44d0-914d-7830adb9f120 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:13.732835 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff4922-3a31-4f0f-96c1-d7a4c53f1d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:13.733526 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff4922-3a31-4f0f-96c1-d7a4c53f1d40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:13.733803 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff4922-3a31-4f0f-96c1-d7a4c53f1d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:13.734071 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff4922-3a31-4f0f-96c1-d7a4c53f1d40 admin admin] PATH_INFO: `/v3/projects/ec7c11e1f3004151b8922799abff7eb5/users/f9b51edde0a543aaadc7933a8e5cc0d3/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:13.753960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd74f67-f9fa-48d6-887d-958de660164b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:13.762340 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff4922-3a31-4f0f-96c1-d7a4c53f1d40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec7c11e1f3004151b8922799abff7eb5, user_id=f9b51edde0a543aaadc7933a8e5cc0d3, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:13.765300 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff4922-3a31-4f0f-96c1-d7a4c53f1d40 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:13.769995 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f546ae9-d9ba-413c-b854-acaa12261663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:13.770882 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f546ae9-d9ba-413c-b854-acaa12261663 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:13.773104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f546ae9-d9ba-413c-b854-acaa12261663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:13.774263 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f546ae9-d9ba-413c-b854-acaa12261663 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:13.776684 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f546ae9-d9ba-413c-b854-acaa12261663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:13.780263 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f546ae9-d9ba-413c-b854-acaa12261663 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:13.806010 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 995/1982] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:13.807069 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f546ae9-d9ba-413c-b854-acaa12261663 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:13.810116 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5f546ae9-d9ba-413c-b854-acaa12261663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:13.826637 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f546ae9-d9ba-413c-b854-acaa12261663 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:13.834203 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc0ea4-4024-46ee-97de-3282d61f8272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:13.834849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc0ea4-4024-46ee-97de-3282d61f8272 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:13.835571 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc0ea4-4024-46ee-97de-3282d61f8272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:13.836002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc0ea4-4024-46ee-97de-3282d61f8272 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:13.847732 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecc0ea4-4024-46ee-97de-3282d61f8272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:13.847732 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecc0ea4-4024-46ee-97de-3282d61f8272 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:13.852123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 996/1983] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:13.855130 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7451dd-c77c-4041-b525-de73b90224b5 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:13.855828 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7451dd-c77c-4041-b525-de73b90224b5 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:13.856469 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7451dd-c77c-4041-b525-de73b90224b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:13.874576 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 988/1984] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:13 2022] PUT /identity/v3/projects/ec7c11e1f3004151b8922799abff7eb5/users/f9b51edde0a543aaadc7933a8e5cc0d3/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:13.875637 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff4922-3a31-4f0f-96c1-d7a4c53f1d40 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:13.882085 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-7e7451dd-c77c-4041-b525-de73b90224b5 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:13.884589 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7e7451dd-c77c-4041-b525-de73b90224b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:13.994158 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 997/1985] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:13.995799 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7451dd-c77c-4041-b525-de73b90224b5 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:14.008414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff4922-3a31-4f0f-96c1-d7a4c53f1d40 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:14.008603 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7451dd-c77c-4041-b525-de73b90224b5 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:14.015332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082bcb7-514c-4596-9a0f-0731876826b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:14.016965 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082bcb7-514c-4596-9a0f-0731876826b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:14.017401 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082bcb7-514c-4596-9a0f-0731876826b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:14.017846 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e979b45-a265-414f-9662-8e2b0e9d0ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:14.018538 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082bcb7-514c-4596-9a0f-0731876826b3 admin admin] PATH_INFO: `/v3/projects/9243e23fdf0e4b158a3e8496828a1bf7/users/4e93d395efb94602aec63a6efc66048e/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:14.018677 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e979b45-a265-414f-9662-8e2b0e9d0ecf admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:14.018924 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e979b45-a265-414f-9662-8e2b0e9d0ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:14.019384 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e979b45-a265-414f-9662-8e2b0e9d0ecf admin admin] PATH_INFO: `/v3/projects/fc0d7968fa32432f94541f49463cb711/users/e9685b79f0bc4c64b06ba55bad670b79/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:14.034707 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6082bcb7-514c-4596-9a0f-0731876826b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9243e23fdf0e4b158a3e8496828a1bf7, user_id=4e93d395efb94602aec63a6efc66048e, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:14.035262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e979b45-a265-414f-9662-8e2b0e9d0ecf admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc0d7968fa32432f94541f49463cb711, user_id=e9685b79f0bc4c64b06ba55bad670b79, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:14.038023 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6082bcb7-514c-4596-9a0f-0731876826b3 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:14.038891 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e979b45-a265-414f-9662-8e2b0e9d0ecf admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:14.103139 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 989/1986] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:13 2022] PUT /identity/v3/projects/9243e23fdf0e4b158a3e8496828a1bf7/users/4e93d395efb94602aec63a6efc66048e/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:14.114204 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082bcb7-514c-4596-9a0f-0731876826b3 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:14.117041 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 998/1987] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:13 2022] PUT /identity/v3/projects/fc0d7968fa32432f94541f49463cb711/users/e9685b79f0bc4c64b06ba55bad670b79/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:14.118180 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e979b45-a265-414f-9662-8e2b0e9d0ecf admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:14.120345 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6e979b45-a265-414f-9662-8e2b0e9d0ecf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:14.185226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e979b45-a265-414f-9662-8e2b0e9d0ecf admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:14.190014 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4fec7-53e2-4bcf-8f82-365806de5339 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:14.190634 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4fec7-53e2-4bcf-8f82-365806de5339 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:14.190917 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4fec7-53e2-4bcf-8f82-365806de5339 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:14.191194 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4fec7-53e2-4bcf-8f82-365806de5339 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:14.191473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082bcb7-514c-4596-9a0f-0731876826b3 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:14.259815 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d4fec7-53e2-4bcf-8f82-365806de5339 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:14.260877 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d078f6e-56a4-4114-8b47-9af047a4e8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:14.261238 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d4fec7-53e2-4bcf-8f82-365806de5339 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:14.261765 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d078f6e-56a4-4114-8b47-9af047a4e8f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:14.262177 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d078f6e-56a4-4114-8b47-9af047a4e8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:14.262576 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d078f6e-56a4-4114-8b47-9af047a4e8f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:14.263545 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d078f6e-56a4-4114-8b47-9af047a4e8f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:14.266064 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d078f6e-56a4-4114-8b47-9af047a4e8f4 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:14.272370 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 999/1988] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:06:14.273809 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede18d7c-6e3e-4ef7-8d7a-3a1ce17092e4 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:14.274277 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede18d7c-6e3e-4ef7-8d7a-3a1ce17092e4 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:14.279330 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede18d7c-6e3e-4ef7-8d7a-3a1ce17092e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:14.285748 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 990/1989] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:14.287832 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d078f6e-56a4-4114-8b47-9af047a4e8f4 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:14.313792 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d078f6e-56a4-4114-8b47-9af047a4e8f4 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:14.318006 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ede18d7c-6e3e-4ef7-8d7a-3a1ce17092e4 None None] MFA Rules not processed for user `4e93d395efb94602aec63a6efc66048e`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:14.319599 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ede18d7c-6e3e-4ef7-8d7a-3a1ce17092e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:14.320471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d689001-8c86-4f06-bd5d-3189d20d729f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:14.321105 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d689001-8c86-4f06-bd5d-3189d20d729f admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:14.321388 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d689001-8c86-4f06-bd5d-3189d20d729f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:14.321660 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d689001-8c86-4f06-bd5d-3189d20d729f admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:14.322572 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d689001-8c86-4f06-bd5d-3189d20d729f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:14.324475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d689001-8c86-4f06-bd5d-3189d20d729f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:14.327880 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:14.347243 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 991/1990] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:14.347243 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d689001-8c86-4f06-bd5d-3189d20d729f admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:14.361557 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d689001-8c86-4f06-bd5d-3189d20d729f admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:14.367410 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a6656-02be-4bfb-81b3-1d966cd81726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:14.368149 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a6656-02be-4bfb-81b3-1d966cd81726 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:14.368570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a6656-02be-4bfb-81b3-1d966cd81726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:14.369286 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a6656-02be-4bfb-81b3-1d966cd81726 admin admin] PATH_INFO: `/v3/projects/ec7c11e1f3004151b8922799abff7eb5/users/f9b51edde0a543aaadc7933a8e5cc0d3/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:14.372404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a6656-02be-4bfb-81b3-1d966cd81726 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec7c11e1f3004151b8922799abff7eb5, user_id=f9b51edde0a543aaadc7933a8e5cc0d3, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:14.376326 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a6656-02be-4bfb-81b3-1d966cd81726 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:14.391238 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 992/1991] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:14 2022] PUT /identity/v3/projects/ec7c11e1f3004151b8922799abff7eb5/users/f9b51edde0a543aaadc7933a8e5cc0d3/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:14.392332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a6656-02be-4bfb-81b3-1d966cd81726 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:14.402211 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1000/1992] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:14.403810 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861cec1-c5ed-45fe-bfbb-b0c1155a6611 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:14.404136 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861cec1-c5ed-45fe-bfbb-b0c1155a6611 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:14.404421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861cec1-c5ed-45fe-bfbb-b0c1155a6611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:14.436872 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b861cec1-c5ed-45fe-bfbb-b0c1155a6611 None None] MFA Rules not processed for user `f9b51edde0a543aaadc7933a8e5cc0d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:14.438129 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b861cec1-c5ed-45fe-bfbb-b0c1155a6611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:14.469601 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a6656-02be-4bfb-81b3-1d966cd81726 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:14.469601 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05fa7c-0f94-4fe4-b4ab-b7249e208ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:14.469601 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05fa7c-0f94-4fe4-b4ab-b7249e208ddb admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:14.469601 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05fa7c-0f94-4fe4-b4ab-b7249e208ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:14.469979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05fa7c-0f94-4fe4-b4ab-b7249e208ddb admin admin] PATH_INFO: `/v3/projects/fc0d7968fa32432f94541f49463cb711/users/e9685b79f0bc4c64b06ba55bad670b79/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:14.470760 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05fa7c-0f94-4fe4-b4ab-b7249e208ddb admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc0d7968fa32432f94541f49463cb711, user_id=e9685b79f0bc4c64b06ba55bad670b79, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:14.473466 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05fa7c-0f94-4fe4-b4ab-b7249e208ddb admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:14.484287 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 993/1993] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:14 2022] PUT /identity/v3/projects/fc0d7968fa32432f94541f49463cb711/users/e9685b79f0bc4c64b06ba55bad670b79/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:14.485162 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05fa7c-0f94-4fe4-b4ab-b7249e208ddb admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:14.489573 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-6e05fa7c-0f94-4fe4-b4ab-b7249e208ddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:14.564119 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1001/1994] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:14 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:14.566434 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077441ad-e8f0-414d-a469-c9c34220cb9c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:14.567896 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077441ad-e8f0-414d-a469-c9c34220cb9c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:14.570273 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077441ad-e8f0-414d-a469-c9c34220cb9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:14.612419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05fa7c-0f94-4fe4-b4ab-b7249e208ddb admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:14.618813 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5076cd-baf8-48cc-be47-cf9af19b1731 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:14.623273 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5076cd-baf8-48cc-be47-cf9af19b1731 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:14.623724 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5076cd-baf8-48cc-be47-cf9af19b1731 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:14.624211 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5076cd-baf8-48cc-be47-cf9af19b1731 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:14.641866 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:14.681395 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-077441ad-e8f0-414d-a469-c9c34220cb9c None None] MFA Rules not processed for user `e9685b79f0bc4c64b06ba55bad670b79`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:14.682828 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-077441ad-e8f0-414d-a469-c9c34220cb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:14.734822 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5076cd-baf8-48cc-be47-cf9af19b1731 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:14.736804 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5076cd-baf8-48cc-be47-cf9af19b1731 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:14.748806 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 994/1995] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:06:14.749850 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5076cd-baf8-48cc-be47-cf9af19b1731 service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:14.753236 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3a5076cd-baf8-48cc-be47-cf9af19b1731 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:14.770621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5076cd-baf8-48cc-be47-cf9af19b1731 service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:14.770621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7633c7b2-a83d-498a-b981-6cdab39eb753 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:14.770621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633c7b2-a83d-498a-b981-6cdab39eb753 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:14.770621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633c7b2-a83d-498a-b981-6cdab39eb753 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:14.771233 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633c7b2-a83d-498a-b981-6cdab39eb753 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:14.794210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1002/1996] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:14 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:14.794210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077441ad-e8f0-414d-a469-c9c34220cb9c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:14.848843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7633c7b2-a83d-498a-b981-6cdab39eb753 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:14.850509 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7633c7b2-a83d-498a-b981-6cdab39eb753 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:14.865082 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 995/1997] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:14 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:06:14.866819 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7633c7b2-a83d-498a-b981-6cdab39eb753 service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:14.879913 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7633c7b2-a83d-498a-b981-6cdab39eb753 service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:14.885736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f2e24-c27f-42b6-b872-893106b6be0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:14.886826 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f2e24-c27f-42b6-b872-893106b6be0f service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:14.887737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f2e24-c27f-42b6-b872-893106b6be0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:14.888259 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f2e24-c27f-42b6-b872-893106b6be0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:14.891229 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077441ad-e8f0-414d-a469-c9c34220cb9c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:14.899128 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cdb773-f51c-487d-b81b-529a668ac8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:14.900176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0f2e24-c27f-42b6-b872-893106b6be0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:14.900287 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cdb773-f51c-487d-b81b-529a668ac8cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:14.900287 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cdb773-f51c-487d-b81b-529a668ac8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:14.900512 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cdb773-f51c-487d-b81b-529a668ac8cf admin admin] PATH_INFO: `/v3/users/4e93d395efb94602aec63a6efc66048e` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:14.901538 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0f2e24-c27f-42b6-b872-893106b6be0f service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:14.901800 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cdb773-f51c-487d-b81b-529a668ac8cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e93d395efb94602aec63a6efc66048e)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:14.903948 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cdb773-f51c-487d-b81b-529a668ac8cf admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:14.912685 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 996/1998] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:14 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:06:14.913449 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f2e24-c27f-42b6-b872-893106b6be0f service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:14.914853 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2b0f2e24-c27f-42b6-b872-893106b6be0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:14.936226 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f2e24-c27f-42b6-b872-893106b6be0f service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:14.950841 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d825f-cee3-4796-b3e3-f28b0536a057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:14.951805 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d825f-cee3-4796-b3e3-f28b0536a057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:14.952173 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d825f-cee3-4796-b3e3-f28b0536a057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:14.952502 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d825f-cee3-4796-b3e3-f28b0536a057 admin admin] PATH_INFO: `/v3/projects/b54ac4223a9245d690a7841f52c31029` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:14.954036 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d825f-cee3-4796-b3e3-f28b0536a057 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b54ac4223a9245d690a7841f52c31029)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:14.957162 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d825f-cee3-4796-b3e3-f28b0536a057 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:15.148202 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-634d825f-cee3-4796-b3e3-f28b0536a057 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b54ac4223a9245d690a7841f52c31029', 'request_id': 'req-634d825f-cee3-4796-b3e3-f28b0536a057'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:15.166384 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-76cdb773-f51c-487d-b81b-529a668ac8cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e93d395efb94602aec63a6efc66048e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:15.170484 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-76cdb773-f51c-487d-b81b-529a668ac8cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e93d395efb94602aec63a6efc66048e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:15.170484 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-76cdb773-f51c-487d-b81b-529a668ac8cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e93d395efb94602aec63a6efc66048e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:15.176473 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-76cdb773-f51c-487d-b81b-529a668ac8cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e93d395efb94602aec63a6efc66048e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:15.190555 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-76cdb773-f51c-487d-b81b-529a668ac8cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e93d395efb94602aec63a6efc66048e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:15.241426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-634d825f-cee3-4796-b3e3-f28b0536a057 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b54ac4223a9245d690a7841f52c31029', 'request_id': 'req-634d825f-cee3-4796-b3e3-f28b0536a057'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:15.248314 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-634d825f-cee3-4796-b3e3-f28b0536a057 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b54ac4223a9245d690a7841f52c31029', 'request_id': 'req-634d825f-cee3-4796-b3e3-f28b0536a057'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:15.258512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-634d825f-cee3-4796-b3e3-f28b0536a057 admin admin] The token cache is being invalidate because project b54ac4223a9245d690a7841f52c31029 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:06:15.258804 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-634d825f-cee3-4796-b3e3-f28b0536a057 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:15.259453 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-634d825f-cee3-4796-b3e3-f28b0536a057 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:15.261002 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 997/1999] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:06:14 2022] DELETE /identity/v3/projects/b54ac4223a9245d690a7841f52c31029 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:15.261767 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d825f-cee3-4796-b3e3-f28b0536a057 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:15.263039 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-634d825f-cee3-4796-b3e3-f28b0536a057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:15.308570 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1003/2000] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:06:14 2022] DELETE /identity/v3/users/4e93d395efb94602aec63a6efc66048e => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:15.309386 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cdb773-f51c-487d-b81b-529a668ac8cf admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:15.310754 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-76cdb773-f51c-487d-b81b-529a668ac8cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:15.320678 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d825f-cee3-4796-b3e3-f28b0536a057 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:15.380478 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df48dc0-caac-4b59-ab07-d29a1440d770 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:15.381788 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df48dc0-caac-4b59-ab07-d29a1440d770 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:15.382315 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df48dc0-caac-4b59-ab07-d29a1440d770 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:15.382803 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df48dc0-caac-4b59-ab07-d29a1440d770 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:15.385685 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-1df48dc0-caac-4b59-ab07-d29a1440d770 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:15.402627 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cdb773-f51c-487d-b81b-529a668ac8cf admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:15.406608 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9933481-db3d-47ef-b6d5-297d59d8b665 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:15.407894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9933481-db3d-47ef-b6d5-297d59d8b665 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:15.407894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9933481-db3d-47ef-b6d5-297d59d8b665 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:15.407894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9933481-db3d-47ef-b6d5-297d59d8b665 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:15.410278 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f9933481-db3d-47ef-b6d5-297d59d8b665 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:15.449306 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:15.464564 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df48dc0-caac-4b59-ab07-d29a1440d770 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:15.467042 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df48dc0-caac-4b59-ab07-d29a1440d770 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:15.471218 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9933481-db3d-47ef-b6d5-297d59d8b665 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:15.472724 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9933481-db3d-47ef-b6d5-297d59d8b665 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:15.482633 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 998/2002] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:06:15.482791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1004/2001] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:06:15.483413 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df48dc0-caac-4b59-ab07-d29a1440d770 service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:15.485141 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363154d0-dadf-4dc8-9028-12c0842d630f None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:15.485141 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363154d0-dadf-4dc8-9028-12c0842d630f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:15.485141 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363154d0-dadf-4dc8-9028-12c0842d630f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:15.485390 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-1df48dc0-caac-4b59-ab07-d29a1440d770 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:15.520361 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-363154d0-dadf-4dc8-9028-12c0842d630f None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:15.521642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-363154d0-dadf-4dc8-9028-12c0842d630f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:15.583082 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df48dc0-caac-4b59-ab07-d29a1440d770 service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:15.588902 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e500832-e417-4627-bbc4-c58c335e9eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:15.589545 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e500832-e417-4627-bbc4-c58c335e9eb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:15.589832 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e500832-e417-4627-bbc4-c58c335e9eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:15.590103 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e500832-e417-4627-bbc4-c58c335e9eb7 admin admin] PATH_INFO: `/v3/users/d6a927f4c03041b1b2996368f7a3fa80` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:15.591286 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e500832-e417-4627-bbc4-c58c335e9eb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6a927f4c03041b1b2996368f7a3fa80)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:15.593514 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e500832-e417-4627-bbc4-c58c335e9eb7 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:15.595372 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1005/2003] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:15.601414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:15.606678 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363154d0-dadf-4dc8-9028-12c0842d630f None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:15.608357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-363154d0-dadf-4dc8-9028-12c0842d630f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:15.670839 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363154d0-dadf-4dc8-9028-12c0842d630f None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:15.674529 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5958d89-122c-4ea2-ba2a-d3dcd20d4f01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:15.675139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5958d89-122c-4ea2-ba2a-d3dcd20d4f01 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:15.675421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5958d89-122c-4ea2-ba2a-d3dcd20d4f01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:15.675696 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5958d89-122c-4ea2-ba2a-d3dcd20d4f01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:15.685061 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5958d89-122c-4ea2-ba2a-d3dcd20d4f01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:15.686535 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5958d89-122c-4ea2-ba2a-d3dcd20d4f01 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:15.696661 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1006/2004] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:06:15.771006 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0e500832-e417-4627-bbc4-c58c335e9eb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6a927f4c03041b1b2996368f7a3fa80'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:15.773207 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0e500832-e417-4627-bbc4-c58c335e9eb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6a927f4c03041b1b2996368f7a3fa80'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:15.786028 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0e500832-e417-4627-bbc4-c58c335e9eb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6a927f4c03041b1b2996368f7a3fa80'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:15.800351 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0e500832-e417-4627-bbc4-c58c335e9eb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6a927f4c03041b1b2996368f7a3fa80'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:15.801100 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0e500832-e417-4627-bbc4-c58c335e9eb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6a927f4c03041b1b2996368f7a3fa80'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:15.893164 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 999/2005] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:06:15 2022] DELETE /identity/v3/users/d6a927f4c03041b1b2996368f7a3fa80 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:16.226468 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5958d89-122c-4ea2-ba2a-d3dcd20d4f01 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:16.228506 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c5958d89-122c-4ea2-ba2a-d3dcd20d4f01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:16.257952 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5958d89-122c-4ea2-ba2a-d3dcd20d4f01 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:16.327341 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f1c761-4c70-4f06-b10c-d6c7d6d9c2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:16.328325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1c761-4c70-4f06-b10c-d6c7d6d9c2c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:16.328924 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1c761-4c70-4f06-b10c-d6c7d6d9c2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:16.329687 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1c761-4c70-4f06-b10c-d6c7d6d9c2c7 admin admin] PATH_INFO: `/v3/projects/9243e23fdf0e4b158a3e8496828a1bf7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:16.331605 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f1c761-4c70-4f06-b10c-d6c7d6d9c2c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9243e23fdf0e4b158a3e8496828a1bf7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:16.334070 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f1c761-4c70-4f06-b10c-d6c7d6d9c2c7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:16.489399 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-00f1c761-4c70-4f06-b10c-d6c7d6d9c2c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9243e23fdf0e4b158a3e8496828a1bf7', 'request_id': 'req-00f1c761-4c70-4f06-b10c-d6c7d6d9c2c7'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:16.493187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:16.509441 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-00f1c761-4c70-4f06-b10c-d6c7d6d9c2c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9243e23fdf0e4b158a3e8496828a1bf7', 'request_id': 'req-00f1c761-4c70-4f06-b10c-d6c7d6d9c2c7'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:16.625157 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-00f1c761-4c70-4f06-b10c-d6c7d6d9c2c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9243e23fdf0e4b158a3e8496828a1bf7', 'request_id': 'req-00f1c761-4c70-4f06-b10c-d6c7d6d9c2c7'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:16.643499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-00f1c761-4c70-4f06-b10c-d6c7d6d9c2c7 admin admin] The token cache is being invalidate because project 9243e23fdf0e4b158a3e8496828a1bf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:06:16.644015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-00f1c761-4c70-4f06-b10c-d6c7d6d9c2c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:16.646681 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-00f1c761-4c70-4f06-b10c-d6c7d6d9c2c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:16.655384 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1007/2006] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:06:16 2022] DELETE /identity/v3/projects/9243e23fdf0e4b158a3e8496828a1bf7 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:16.664935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399bdc7-17ea-47e9-8b69-ce9084d5494c None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:16.665335 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399bdc7-17ea-47e9-8b69-ce9084d5494c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:16.665684 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399bdc7-17ea-47e9-8b69-ce9084d5494c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:16.692630 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-d399bdc7-17ea-47e9-8b69-ce9084d5494c None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:16.694465 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d399bdc7-17ea-47e9-8b69-ce9084d5494c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:16.795614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1000/2007] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:16.804465 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957474b-215d-4383-821e-87171843657a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:16.805009 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957474b-215d-4383-821e-87171843657a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:16.805009 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957474b-215d-4383-821e-87171843657a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:16.835668 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399bdc7-17ea-47e9-8b69-ce9084d5494c None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:16.837150 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d399bdc7-17ea-47e9-8b69-ce9084d5494c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:16.838755 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-4957474b-215d-4383-821e-87171843657a None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:16.840317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4957474b-215d-4383-821e-87171843657a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:16.849234 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399bdc7-17ea-47e9-8b69-ce9084d5494c None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:16.855048 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ac1b8-12a8-4419-92b7-c537cee6049f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:16.857664 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ac1b8-12a8-4419-92b7-c537cee6049f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:16.857664 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ac1b8-12a8-4419-92b7-c537cee6049f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:16.857664 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ac1b8-12a8-4419-92b7-c537cee6049f admin admin] PATH_INFO: `/v3/projects/80069bd1dee942c7b4af1d06a3efa656` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:16.857664 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ac1b8-12a8-4419-92b7-c537cee6049f admin admin] RBAC: Authorizing `identity:delete_project(project_id=80069bd1dee942c7b4af1d06a3efa656)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:16.859519 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ac1b8-12a8-4419-92b7-c537cee6049f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:16.864297 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1008/2008] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:16.868843 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4957474b-215d-4383-821e-87171843657a None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:16.889118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4957474b-215d-4383-821e-87171843657a None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:16.890979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27724c8d-5d12-4f65-9012-6419571533d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:16.892155 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27724c8d-5d12-4f65-9012-6419571533d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:16.892610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27724c8d-5d12-4f65-9012-6419571533d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:16.893208 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27724c8d-5d12-4f65-9012-6419571533d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:16.894014 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27724c8d-5d12-4f65-9012-6419571533d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:06:16.894467 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27724c8d-5d12-4f65-9012-6419571533d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:16.896796 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27724c8d-5d12-4f65-9012-6419571533d8 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:16.929951 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:16.940991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1009/2009] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:06:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:06:16.942043 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27724c8d-5d12-4f65-9012-6419571533d8 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:16.955381 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27724c8d-5d12-4f65-9012-6419571533d8 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:16.963291 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb470d-1900-4cec-91f7-3179c1ffe469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:16.964145 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb470d-1900-4cec-91f7-3179c1ffe469 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:16.964624 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb470d-1900-4cec-91f7-3179c1ffe469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:16.964903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb470d-1900-4cec-91f7-3179c1ffe469 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:16.965894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bb470d-1900-4cec-91f7-3179c1ffe469 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:16.967861 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bb470d-1900-4cec-91f7-3179c1ffe469 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:17.011066 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3b2ac1b8-12a8-4419-92b7-c537cee6049f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80069bd1dee942c7b4af1d06a3efa656', 'request_id': 'req-3b2ac1b8-12a8-4419-92b7-c537cee6049f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:17.119522 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3b2ac1b8-12a8-4419-92b7-c537cee6049f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80069bd1dee942c7b4af1d06a3efa656', 'request_id': 'req-3b2ac1b8-12a8-4419-92b7-c537cee6049f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:17.128432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1010/2010] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:06:16 2022] POST /identity/v3/projects => generated 371 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:17.130427 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb470d-1900-4cec-91f7-3179c1ffe469 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:17.132809 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3b2ac1b8-12a8-4419-92b7-c537cee6049f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80069bd1dee942c7b4af1d06a3efa656', 'request_id': 'req-3b2ac1b8-12a8-4419-92b7-c537cee6049f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:17.152600 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3b2ac1b8-12a8-4419-92b7-c537cee6049f admin admin] The token cache is being invalidate because project 80069bd1dee942c7b4af1d06a3efa656 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:06:17.154509 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3b2ac1b8-12a8-4419-92b7-c537cee6049f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:17.155968 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3b2ac1b8-12a8-4419-92b7-c537cee6049f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:17.161550 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1001/2011] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:06:16 2022] DELETE /identity/v3/projects/80069bd1dee942c7b4af1d06a3efa656 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:17.166010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ac1b8-12a8-4419-92b7-c537cee6049f admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:17.167389 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3b2ac1b8-12a8-4419-92b7-c537cee6049f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:17.220384 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb470d-1900-4cec-91f7-3179c1ffe469 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:17.225971 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad9a052-f164-4f3a-aa77-ca12b00dfe11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:17.226584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9a052-f164-4f3a-aa77-ca12b00dfe11 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:17.226863 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9a052-f164-4f3a-aa77-ca12b00dfe11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:17.227135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9a052-f164-4f3a-aa77-ca12b00dfe11 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:17.227921 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad9a052-f164-4f3a-aa77-ca12b00dfe11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:17.229885 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad9a052-f164-4f3a-aa77-ca12b00dfe11 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:17.234761 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ac1b8-12a8-4419-92b7-c537cee6049f admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:17.239397 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595fd1a-9f3f-46ef-b4a9-a6220117bc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:17.240146 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595fd1a-9f3f-46ef-b4a9-a6220117bc44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:17.240451 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595fd1a-9f3f-46ef-b4a9-a6220117bc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:17.240721 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595fd1a-9f3f-46ef-b4a9-a6220117bc44 admin admin] PATH_INFO: `/v3/users/e7f84d3eb63c4e7b87883de16a99db9c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:17.241846 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595fd1a-9f3f-46ef-b4a9-a6220117bc44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7f84d3eb63c4e7b87883de16a99db9c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:17.243754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595fd1a-9f3f-46ef-b4a9-a6220117bc44 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:17.325322 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1011/2012] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:06:17 2022] POST /identity/v3/users => generated 286 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:17.329506 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad9a052-f164-4f3a-aa77-ca12b00dfe11 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:17.331058 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bad9a052-f164-4f3a-aa77-ca12b00dfe11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:17.343003 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad9a052-f164-4f3a-aa77-ca12b00dfe11 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:17.347514 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542e994-ae05-412d-8f04-899759a2c7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:17.348098 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542e994-ae05-412d-8f04-899759a2c7c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:17.350205 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542e994-ae05-412d-8f04-899759a2c7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:17.350499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542e994-ae05-412d-8f04-899759a2c7c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:17.351149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b542e994-ae05-412d-8f04-899759a2c7c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:17.352825 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b542e994-ae05-412d-8f04-899759a2c7c8 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:17.362898 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1012/2013] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:06:17.366883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542e994-ae05-412d-8f04-899759a2c7c8 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:17.376405 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542e994-ae05-412d-8f04-899759a2c7c8 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:17.381151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f6b72-a090-4f68-a134-804f1b2cda36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:17.381745 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f6b72-a090-4f68-a134-804f1b2cda36 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:17.387246 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f6b72-a090-4f68-a134-804f1b2cda36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:17.387246 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f6b72-a090-4f68-a134-804f1b2cda36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:17.387246 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f6b72-a090-4f68-a134-804f1b2cda36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:17.387246 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f6b72-a090-4f68-a134-804f1b2cda36 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:17.395935 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1013/2014] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:06:17.401049 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f6b72-a090-4f68-a134-804f1b2cda36 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:17.410676 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f6b72-a090-4f68-a134-804f1b2cda36 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:17.416253 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f37a1-1a84-408e-b487-e93f0940c858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:17.416902 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f37a1-1a84-408e-b487-e93f0940c858 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:17.421751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f37a1-1a84-408e-b487-e93f0940c858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:17.421751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f37a1-1a84-408e-b487-e93f0940c858 admin admin] PATH_INFO: `/v3/projects/a5770a06426f46c3a5761a9598d832eb/users/a1aa259d6dd94936a45394f0a4812e3a/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:17.432431 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16f37a1-1a84-408e-b487-e93f0940c858 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5770a06426f46c3a5761a9598d832eb, user_id=a1aa259d6dd94936a45394f0a4812e3a, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:17.435090 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16f37a1-1a84-408e-b487-e93f0940c858 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:17.455749 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a595fd1a-9f3f-46ef-b4a9-a6220117bc44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7f84d3eb63c4e7b87883de16a99db9c'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:17.456466 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a595fd1a-9f3f-46ef-b4a9-a6220117bc44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7f84d3eb63c4e7b87883de16a99db9c'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:17.464035 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a595fd1a-9f3f-46ef-b4a9-a6220117bc44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7f84d3eb63c4e7b87883de16a99db9c'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:17.475870 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a595fd1a-9f3f-46ef-b4a9-a6220117bc44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7f84d3eb63c4e7b87883de16a99db9c'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:17.476558 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a595fd1a-9f3f-46ef-b4a9-a6220117bc44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7f84d3eb63c4e7b87883de16a99db9c'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:17.510113 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1014/2015] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:17 2022] PUT /identity/v3/projects/a5770a06426f46c3a5761a9598d832eb/users/a1aa259d6dd94936a45394f0a4812e3a/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:06:17.519981 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f2340-3534-4904-902a-06f4a6b2e5de None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:17.520296 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f2340-3534-4904-902a-06f4a6b2e5de None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:17.520571 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f2340-3534-4904-902a-06f4a6b2e5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:17.548709 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ab8f2340-3534-4904-902a-06f4a6b2e5de None None] MFA Rules not processed for user `a1aa259d6dd94936a45394f0a4812e3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:17.549741 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ab8f2340-3534-4904-902a-06f4a6b2e5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:17.579334 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1002/2016] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:06:17 2022] DELETE /identity/v3/users/e7f84d3eb63c4e7b87883de16a99db9c => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:06:17.615848 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1015/2017] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:06:17.634105 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595fd1a-9f3f-46ef-b4a9-a6220117bc44 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:17.635713 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a595fd1a-9f3f-46ef-b4a9-a6220117bc44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:17.711032 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595fd1a-9f3f-46ef-b4a9-a6220117bc44 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:17.715035 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58a1d5-c8c4-485b-9cad-628d03de3aa5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:17.715650 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58a1d5-c8c4-485b-9cad-628d03de3aa5 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:17.715934 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58a1d5-c8c4-485b-9cad-628d03de3aa5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:17.716258 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58a1d5-c8c4-485b-9cad-628d03de3aa5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:17.778217 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58a1d5-c8c4-485b-9cad-628d03de3aa5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:17.779669 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58a1d5-c8c4-485b-9cad-628d03de3aa5 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:17.796812 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1003/2018] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:06:17.829199 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f2340-3534-4904-902a-06f4a6b2e5de None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:17.830851 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ab8f2340-3534-4904-902a-06f4a6b2e5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:17.909499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f2340-3534-4904-902a-06f4a6b2e5de None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:17.984863 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f827527-1880-4685-8a33-312416eedda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:17.985518 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f827527-1880-4685-8a33-312416eedda1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:17.986156 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f827527-1880-4685-8a33-312416eedda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:17.986156 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f827527-1880-4685-8a33-312416eedda1 admin admin] PATH_INFO: `/v3/users/a1aa259d6dd94936a45394f0a4812e3a` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:17.987314 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f827527-1880-4685-8a33-312416eedda1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1aa259d6dd94936a45394f0a4812e3a)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:17.989359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f827527-1880-4685-8a33-312416eedda1 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:18.170495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58a1d5-c8c4-485b-9cad-628d03de3aa5 service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:18.170495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5c58a1d5-c8c4-485b-9cad-628d03de3aa5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:18.187423 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58a1d5-c8c4-485b-9cad-628d03de3aa5 service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:18.194891 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccab823-7209-4b2f-b32e-88f860778418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:18.195545 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccab823-7209-4b2f-b32e-88f860778418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:18.195845 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccab823-7209-4b2f-b32e-88f860778418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:18.196127 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccab823-7209-4b2f-b32e-88f860778418 admin admin] PATH_INFO: `/v3/projects/6ffd4e04cb794d58be2375b023f3ae8f` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:18.197279 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccab823-7209-4b2f-b32e-88f860778418 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ffd4e04cb794d58be2375b023f3ae8f)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:18.199246 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccab823-7209-4b2f-b32e-88f860778418 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:18.212888 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2f827527-1880-4685-8a33-312416eedda1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1aa259d6dd94936a45394f0a4812e3a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:18.213630 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2f827527-1880-4685-8a33-312416eedda1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1aa259d6dd94936a45394f0a4812e3a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:18.214240 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2f827527-1880-4685-8a33-312416eedda1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1aa259d6dd94936a45394f0a4812e3a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:18.221515 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2f827527-1880-4685-8a33-312416eedda1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1aa259d6dd94936a45394f0a4812e3a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:18.243563 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2f827527-1880-4685-8a33-312416eedda1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1aa259d6dd94936a45394f0a4812e3a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:18.253862 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:18.323357 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7ccab823-7209-4b2f-b32e-88f860778418 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ffd4e04cb794d58be2375b023f3ae8f', 'request_id': 'req-7ccab823-7209-4b2f-b32e-88f860778418'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:18.340140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1016/2019] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:06:17 2022] DELETE /identity/v3/users/a1aa259d6dd94936a45394f0a4812e3a => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:18.344584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea93c6d-f6b9-4c28-a5f6-0c56dd02bd19 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:18.344878 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea93c6d-f6b9-4c28-a5f6-0c56dd02bd19 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:18.345204 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea93c6d-f6b9-4c28-a5f6-0c56dd02bd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:18.367302 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-6ea93c6d-f6b9-4c28-a5f6-0c56dd02bd19 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:18.368545 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6ea93c6d-f6b9-4c28-a5f6-0c56dd02bd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:18.413678 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7ccab823-7209-4b2f-b32e-88f860778418 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ffd4e04cb794d58be2375b023f3ae8f', 'request_id': 'req-7ccab823-7209-4b2f-b32e-88f860778418'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:18.421805 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7ccab823-7209-4b2f-b32e-88f860778418 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ffd4e04cb794d58be2375b023f3ae8f', 'request_id': 'req-7ccab823-7209-4b2f-b32e-88f860778418'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:18.435193 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7ccab823-7209-4b2f-b32e-88f860778418 admin admin] The token cache is being invalidate because project 6ffd4e04cb794d58be2375b023f3ae8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:06:18.435489 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7ccab823-7209-4b2f-b32e-88f860778418 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:18.436278 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7ccab823-7209-4b2f-b32e-88f860778418 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:18.437921 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1004/2020] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:06:18 2022] DELETE /identity/v3/projects/6ffd4e04cb794d58be2375b023f3ae8f => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:06:18.443733 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1017/2021] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:06:18.445149 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3d1c6-c423-4dd8-822e-7d74e53d13bd None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:18.462793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3d1c6-c423-4dd8-822e-7d74e53d13bd None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:18.462793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3d1c6-c423-4dd8-822e-7d74e53d13bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:18.467114 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-6ea3d1c6-c423-4dd8-822e-7d74e53d13bd None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:18.467267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-6ea3d1c6-c423-4dd8-822e-7d74e53d13bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:18.471565 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea93c6d-f6b9-4c28-a5f6-0c56dd02bd19 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:18.473054 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6ea93c6d-f6b9-4c28-a5f6-0c56dd02bd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:18.476044 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1005/2022] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:06:18.484266 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25597f06-9729-4260-b962-5337e38c4700 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:18.484559 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25597f06-9729-4260-b962-5337e38c4700 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:18.484835 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25597f06-9729-4260-b962-5337e38c4700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:18.504888 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-25597f06-9729-4260-b962-5337e38c4700 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:18.506151 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-25597f06-9729-4260-b962-5337e38c4700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:18.511485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:18.515270 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1006/2023] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:06:18.519444 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25597f06-9729-4260-b962-5337e38c4700 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:18.528685 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25597f06-9729-4260-b962-5337e38c4700 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:18.533565 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be6bcf-30a8-4b63-95a0-f7dc78b87c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:18.534197 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be6bcf-30a8-4b63-95a0-f7dc78b87c51 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:18.537819 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be6bcf-30a8-4b63-95a0-f7dc78b87c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:18.537819 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be6bcf-30a8-4b63-95a0-f7dc78b87c51 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:18.537819 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be6bcf-30a8-4b63-95a0-f7dc78b87c51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:06:18.537819 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be6bcf-30a8-4b63-95a0-f7dc78b87c51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:18.538251 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be6bcf-30a8-4b63-95a0-f7dc78b87c51 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:18.538755 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea93c6d-f6b9-4c28-a5f6-0c56dd02bd19 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:18.542465 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b60e1c-7503-45c4-afb2-ed24c5f84b32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:18.543069 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b60e1c-7503-45c4-afb2-ed24c5f84b32 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:18.543344 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b60e1c-7503-45c4-afb2-ed24c5f84b32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:18.543610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b60e1c-7503-45c4-afb2-ed24c5f84b32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:18.553840 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1007/2024] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:06:18.553985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b60e1c-7503-45c4-afb2-ed24c5f84b32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:18.554777 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b60e1c-7503-45c4-afb2-ed24c5f84b32 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:18.556729 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be6bcf-30a8-4b63-95a0-f7dc78b87c51 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:18.565325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1018/2025] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:06:18.566883 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be6bcf-30a8-4b63-95a0-f7dc78b87c51 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:18.572124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f1627-c5df-4a63-aaad-d481ec0be15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:18.572838 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f1627-c5df-4a63-aaad-d481ec0be15f admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:18.573105 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f1627-c5df-4a63-aaad-d481ec0be15f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:18.573374 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f1627-c5df-4a63-aaad-d481ec0be15f admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:18.574008 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f1627-c5df-4a63-aaad-d481ec0be15f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:18.576000 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f1627-c5df-4a63-aaad-d481ec0be15f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:18.649125 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b60e1c-7503-45c4-afb2-ed24c5f84b32 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:18.649279 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1008/2026] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:06:18 2022] POST /identity/v3/projects => generated 385 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:18.660191 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b60e1c-7503-45c4-afb2-ed24c5f84b32 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:18.721339 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1138f-b142-43ca-9b75-08e93c19777b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:18.721955 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1138f-b142-43ca-9b75-08e93c19777b admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:18.722234 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1138f-b142-43ca-9b75-08e93c19777b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:18.722507 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1138f-b142-43ca-9b75-08e93c19777b admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:18.723236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1138f-b142-43ca-9b75-08e93c19777b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:18.725795 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1138f-b142-43ca-9b75-08e93c19777b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:18.831565 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1019/2027] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:06:18 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:06:18.838678 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f1627-c5df-4a63-aaad-d481ec0be15f admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:18.862912 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f1627-c5df-4a63-aaad-d481ec0be15f admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:18.872381 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba73fa-810c-4e3f-ba69-911d7d6f308a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:18.873207 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba73fa-810c-4e3f-ba69-911d7d6f308a admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:18.873496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba73fa-810c-4e3f-ba69-911d7d6f308a admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:18.874438 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba73fa-810c-4e3f-ba69-911d7d6f308a admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:18.881091 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba73fa-810c-4e3f-ba69-911d7d6f308a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:18.881091 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba73fa-810c-4e3f-ba69-911d7d6f308a admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:18.902172 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1009/2028] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:18.915663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1138f-b142-43ca-9b75-08e93c19777b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:18.933683 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1138f-b142-43ca-9b75-08e93c19777b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:18.940702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89973aa-6a77-454a-a304-2d1a15713e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:18.942346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89973aa-6a77-454a-a304-2d1a15713e64 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:18.942829 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89973aa-6a77-454a-a304-2d1a15713e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:18.945908 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89973aa-6a77-454a-a304-2d1a15713e64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:18.946998 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89973aa-6a77-454a-a304-2d1a15713e64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:18.949524 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89973aa-6a77-454a-a304-2d1a15713e64 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:18.967567 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1020/2029] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:18.971362 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba73fa-810c-4e3f-ba69-911d7d6f308a admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:19.005082 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba73fa-810c-4e3f-ba69-911d7d6f308a admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:19.020985 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e8dee-25da-47b6-affd-69aaab5692bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:19.021594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e8dee-25da-47b6-affd-69aaab5692bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:19.022911 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e8dee-25da-47b6-affd-69aaab5692bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:19.023428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e8dee-25da-47b6-affd-69aaab5692bf admin admin] PATH_INFO: `/v3/projects/d31d8ff2123a4119837f06a5120d15b5/users/5e71d02e626f42bda29c4f6afe82bb0b/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:19.050195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e8dee-25da-47b6-affd-69aaab5692bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d31d8ff2123a4119837f06a5120d15b5, user_id=5e71d02e626f42bda29c4f6afe82bb0b, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:19.053770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e8dee-25da-47b6-affd-69aaab5692bf admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:19.149326 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1010/2030] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:18 2022] PUT /identity/v3/projects/d31d8ff2123a4119837f06a5120d15b5/users/5e71d02e626f42bda29c4f6afe82bb0b/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:19.160580 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342df378-a55b-43de-887f-b8bc78b00e4b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:19.160876 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342df378-a55b-43de-887f-b8bc78b00e4b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:19.161162 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342df378-a55b-43de-887f-b8bc78b00e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:19.190194 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:19.194537 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-342df378-a55b-43de-887f-b8bc78b00e4b None None] MFA Rules not processed for user `5e71d02e626f42bda29c4f6afe82bb0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:19.195778 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-342df378-a55b-43de-887f-b8bc78b00e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:19.262300 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1021/2031] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:19.266936 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e8dee-25da-47b6-affd-69aaab5692bf admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:19.329047 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342df378-a55b-43de-887f-b8bc78b00e4b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:19.330554 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-342df378-a55b-43de-887f-b8bc78b00e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:19.332212 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e8dee-25da-47b6-affd-69aaab5692bf admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:19.337933 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8acb9e-9682-426f-b600-43f6b313ab01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:19.338552 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8acb9e-9682-426f-b600-43f6b313ab01 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:19.338838 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8acb9e-9682-426f-b600-43f6b313ab01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:19.339141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8acb9e-9682-426f-b600-43f6b313ab01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:19.340439 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8acb9e-9682-426f-b600-43f6b313ab01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:19.342006 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8acb9e-9682-426f-b600-43f6b313ab01 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:19.345609 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342df378-a55b-43de-887f-b8bc78b00e4b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:19.350646 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f919c48-f390-4d83-af05-f401583b4984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:19.351284 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f919c48-f390-4d83-af05-f401583b4984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:19.351701 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f919c48-f390-4d83-af05-f401583b4984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:19.352105 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f919c48-f390-4d83-af05-f401583b4984 admin admin] PATH_INFO: `/v3/projects/a5770a06426f46c3a5761a9598d832eb` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:19.353155 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f919c48-f390-4d83-af05-f401583b4984 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5770a06426f46c3a5761a9598d832eb)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:19.355060 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f919c48-f390-4d83-af05-f401583b4984 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:19.465621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1011/2032] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:06:19 2022] POST /identity/v3/projects => generated 383 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:19.469789 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8acb9e-9682-426f-b600-43f6b313ab01 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:19.471539 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:19.478944 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-3f919c48-f390-4d83-af05-f401583b4984 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5770a06426f46c3a5761a9598d832eb', 'request_id': 'req-3f919c48-f390-4d83-af05-f401583b4984'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:19.485670 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-3f919c48-f390-4d83-af05-f401583b4984 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5770a06426f46c3a5761a9598d832eb', 'request_id': 'req-3f919c48-f390-4d83-af05-f401583b4984'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:19.539371 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8acb9e-9682-426f-b600-43f6b313ab01 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:19.544409 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee87a8-4755-423f-9a03-520ff5aaafe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:19.545112 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee87a8-4755-423f-9a03-520ff5aaafe9 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:19.545394 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee87a8-4755-423f-9a03-520ff5aaafe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:19.545692 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee87a8-4755-423f-9a03-520ff5aaafe9 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:19.546326 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ee87a8-4755-423f-9a03-520ff5aaafe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:19.548525 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ee87a8-4755-423f-9a03-520ff5aaafe9 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:19.618332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-3f919c48-f390-4d83-af05-f401583b4984 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5770a06426f46c3a5761a9598d832eb', 'request_id': 'req-3f919c48-f390-4d83-af05-f401583b4984'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:19.629419 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-3f919c48-f390-4d83-af05-f401583b4984 admin admin] The token cache is being invalidate because project a5770a06426f46c3a5761a9598d832eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:06:19.629791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-3f919c48-f390-4d83-af05-f401583b4984 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:19.630708 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-3f919c48-f390-4d83-af05-f401583b4984 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:19.632985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1022/2033] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:06:19 2022] DELETE /identity/v3/projects/a5770a06426f46c3a5761a9598d832eb => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:19.637861 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c0f4a-e1e1-46e6-8e76-a8c2885a61a7 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:19.638144 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c0f4a-e1e1-46e6-8e76-a8c2885a61a7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:19.638477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c0f4a-e1e1-46e6-8e76-a8c2885a61a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:19.655024 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1012/2034] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:06:19 2022] POST /identity/v3/users => generated 298 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:19.658579 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-543c0f4a-e1e1-46e6-8e76-a8c2885a61a7 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:19.658956 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee87a8-4755-423f-9a03-520ff5aaafe9 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:19.659791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-543c0f4a-e1e1-46e6-8e76-a8c2885a61a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:19.660789 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-25ee87a8-4755-423f-9a03-520ff5aaafe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:19.674698 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1023/2035] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:06:19.675397 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee87a8-4755-423f-9a03-520ff5aaafe9 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:19.676112 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d6c4b-838a-4f83-9e71-0e8360f0c66a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:19.676330 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d6c4b-838a-4f83-9e71-0e8360f0c66a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:19.676632 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d6c4b-838a-4f83-9e71-0e8360f0c66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:19.686385 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e51b2a4-1009-4fe6-978c-ead02d949c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:19.687009 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51b2a4-1009-4fe6-978c-ead02d949c77 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:19.690777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51b2a4-1009-4fe6-978c-ead02d949c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:19.690777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51b2a4-1009-4fe6-978c-ead02d949c77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:19.690777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e51b2a4-1009-4fe6-978c-ead02d949c77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:19.690777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e51b2a4-1009-4fe6-978c-ead02d949c77 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:19.697621 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ef6d6c4b-838a-4f83-9e71-0e8360f0c66a None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:19.698899 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ef6d6c4b-838a-4f83-9e71-0e8360f0c66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:19.705973 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1013/2036] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:06:19.707011 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e51b2a4-1009-4fe6-978c-ead02d949c77 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:19.710780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1024/2037] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:19.715266 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d6c4b-838a-4f83-9e71-0e8360f0c66a None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:19.717608 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e51b2a4-1009-4fe6-978c-ead02d949c77 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:19.723071 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c37250-42f6-40c2-a6e0-95be0cc83f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:19.723735 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c37250-42f6-40c2-a6e0-95be0cc83f9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:19.724019 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c37250-42f6-40c2-a6e0-95be0cc83f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:19.725296 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c37250-42f6-40c2-a6e0-95be0cc83f9b admin admin] PATH_INFO: `/v3/projects/f766d1760282466d8ef637b17c32fc0b/users/a0e0cd355ab247e9b94bc1209753f0a5/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:19.727038 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d6c4b-838a-4f83-9e71-0e8360f0c66a None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:19.732492 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb81bc4-233c-4836-b819-c75e6e978d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:19.733320 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb81bc4-233c-4836-b819-c75e6e978d26 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:19.733601 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb81bc4-233c-4836-b819-c75e6e978d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:19.733872 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb81bc4-233c-4836-b819-c75e6e978d26 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:19.734527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb81bc4-233c-4836-b819-c75e6e978d26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:06:19.734973 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb81bc4-233c-4836-b819-c75e6e978d26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:19.736612 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb81bc4-233c-4836-b819-c75e6e978d26 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:19.742005 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c37250-42f6-40c2-a6e0-95be0cc83f9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f766d1760282466d8ef637b17c32fc0b, user_id=a0e0cd355ab247e9b94bc1209753f0a5, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:19.744916 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c37250-42f6-40c2-a6e0-95be0cc83f9b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:19.749856 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1025/2038] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:06:19.754281 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb81bc4-233c-4836-b819-c75e6e978d26 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:19.764518 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb81bc4-233c-4836-b819-c75e6e978d26 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:19.769327 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e9051b-e862-4c2c-8a6d-85b7b80a61cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:19.769986 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9051b-e862-4c2c-8a6d-85b7b80a61cd admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:19.770270 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9051b-e862-4c2c-8a6d-85b7b80a61cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:19.770545 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9051b-e862-4c2c-8a6d-85b7b80a61cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:19.771177 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e9051b-e862-4c2c-8a6d-85b7b80a61cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:19.773160 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e9051b-e862-4c2c-8a6d-85b7b80a61cd admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:19.847204 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1014/2039] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:19 2022] PUT /identity/v3/projects/f766d1760282466d8ef637b17c32fc0b/users/a0e0cd355ab247e9b94bc1209753f0a5/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:06:19.851725 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c37250-42f6-40c2-a6e0-95be0cc83f9b admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:19.859791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1026/2040] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:06:19 2022] POST /identity/v3/projects => generated 371 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:06:19.863901 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e9051b-e862-4c2c-8a6d-85b7b80a61cd admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:19.913967 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c37250-42f6-40c2-a6e0-95be0cc83f9b admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:19.925828 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e9051b-e862-4c2c-8a6d-85b7b80a61cd admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:19.931305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8136e1-ab7a-463e-9b62-5643638e3699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:19.931905 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8136e1-ab7a-463e-9b62-5643638e3699 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:19.932245 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8136e1-ab7a-463e-9b62-5643638e3699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:19.932518 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8136e1-ab7a-463e-9b62-5643638e3699 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:19.933168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8136e1-ab7a-463e-9b62-5643638e3699 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:19.935090 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8136e1-ab7a-463e-9b62-5643638e3699 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:19.968966 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e1c3b-f396-40ba-8974-9ab09e1aacc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:19.969477 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e1c3b-f396-40ba-8974-9ab09e1aacc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:19.969754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e1c3b-f396-40ba-8974-9ab09e1aacc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:19.970019 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e1c3b-f396-40ba-8974-9ab09e1aacc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:19.970647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e1c3b-f396-40ba-8974-9ab09e1aacc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:19.972298 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e1c3b-f396-40ba-8974-9ab09e1aacc4 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:19.982551 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1015/2041] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:06:19.986658 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e1c3b-f396-40ba-8974-9ab09e1aacc4 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:19.996895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e1c3b-f396-40ba-8974-9ab09e1aacc4 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:20.001490 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f03cd-130c-4ce9-a4e9-3d48080ea4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:20.002111 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f03cd-130c-4ce9-a4e9-3d48080ea4fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:20.002388 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f03cd-130c-4ce9-a4e9-3d48080ea4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:20.002657 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f03cd-130c-4ce9-a4e9-3d48080ea4fe admin admin] PATH_INFO: `/v3/projects/f766d1760282466d8ef637b17c32fc0b/users/a0e0cd355ab247e9b94bc1209753f0a5/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:20.004613 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3f03cd-130c-4ce9-a4e9-3d48080ea4fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=f766d1760282466d8ef637b17c32fc0b, user_id=a0e0cd355ab247e9b94bc1209753f0a5, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:20.012372 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3f03cd-130c-4ce9-a4e9-3d48080ea4fe admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:20.019113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1016/2042] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:19 2022] PUT /identity/v3/projects/f766d1760282466d8ef637b17c32fc0b/users/a0e0cd355ab247e9b94bc1209753f0a5/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:06:20.029158 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d19086-60e4-4f0a-a558-754fdb091b98 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:20.029448 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d19086-60e4-4f0a-a558-754fdb091b98 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:20.029724 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d19086-60e4-4f0a-a558-754fdb091b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:20.042051 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1027/2043] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:06:19 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:06:20.046299 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8136e1-ab7a-463e-9b62-5643638e3699 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:20.057507 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8136e1-ab7a-463e-9b62-5643638e3699 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:20.060013 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-63d19086-60e4-4f0a-a558-754fdb091b98 None None] MFA Rules not processed for user `a0e0cd355ab247e9b94bc1209753f0a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:20.061275 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-63d19086-60e4-4f0a-a558-754fdb091b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:20.116691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba126b-d5f2-45e6-ba9a-b2b75c04d59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:20.117342 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba126b-d5f2-45e6-ba9a-b2b75c04d59e admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:20.117625 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba126b-d5f2-45e6-ba9a-b2b75c04d59e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:20.117900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba126b-d5f2-45e6-ba9a-b2b75c04d59e admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:20.118557 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba126b-d5f2-45e6-ba9a-b2b75c04d59e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:20.120289 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba126b-d5f2-45e6-ba9a-b2b75c04d59e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:20.134932 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1017/2044] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:20 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:06:20.135065 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1028/2045] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:20.135976 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d19086-60e4-4f0a-a558-754fdb091b98 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:20.143307 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba126b-d5f2-45e6-ba9a-b2b75c04d59e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:20.145403 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2aba126b-d5f2-45e6-ba9a-b2b75c04d59e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:20.149030 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d19086-60e4-4f0a-a558-754fdb091b98 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:20.155668 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e5313-ef3c-4863-9d3b-c03f67826e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:20.156358 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e5313-ef3c-4863-9d3b-c03f67826e32 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:20.156666 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e5313-ef3c-4863-9d3b-c03f67826e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:20.156943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e5313-ef3c-4863-9d3b-c03f67826e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:20.157616 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e5313-ef3c-4863-9d3b-c03f67826e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:20.168191 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e5313-ef3c-4863-9d3b-c03f67826e32 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:20.172096 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1018/2046] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:20.176392 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e5313-ef3c-4863-9d3b-c03f67826e32 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:20.191452 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e5313-ef3c-4863-9d3b-c03f67826e32 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:20.197307 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b2c07-ff48-4ab6-a368-5040fb544a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:20.197816 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b2c07-ff48-4ab6-a368-5040fb544a8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:20.198132 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b2c07-ff48-4ab6-a368-5040fb544a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:20.198418 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b2c07-ff48-4ab6-a368-5040fb544a8c admin admin] PATH_INFO: `/v3/projects/d26a3ee560e449b8b1188b9defa98fa6/users/b17adcb2be25427485d5004c7821e094/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:20.222739 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b2c07-ff48-4ab6-a368-5040fb544a8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d26a3ee560e449b8b1188b9defa98fa6, user_id=b17adcb2be25427485d5004c7821e094, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:20.226420 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b2c07-ff48-4ab6-a368-5040fb544a8c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:20.253547 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba126b-d5f2-45e6-ba9a-b2b75c04d59e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:20.261762 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7358a6f-c7e6-4370-9741-10819cceb590 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:20.265591 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7358a6f-c7e6-4370-9741-10819cceb590 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:20.265591 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7358a6f-c7e6-4370-9741-10819cceb590 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:20.265591 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7358a6f-c7e6-4370-9741-10819cceb590 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:20.267785 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f7358a6f-c7e6-4370-9741-10819cceb590 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:20.322986 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1019/2047] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:20 2022] PUT /identity/v3/projects/d26a3ee560e449b8b1188b9defa98fa6/users/b17adcb2be25427485d5004c7821e094/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:20.328278 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c44567-6553-42e7-b88a-d4c9a55d5cd2 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:20.328570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c44567-6553-42e7-b88a-d4c9a55d5cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:20.328844 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c44567-6553-42e7-b88a-d4c9a55d5cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:20.349618 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-f9c44567-6553-42e7-b88a-d4c9a55d5cd2 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:20.351714 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f9c44567-6553-42e7-b88a-d4c9a55d5cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:20.352332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7358a6f-c7e6-4370-9741-10819cceb590 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:20.354047 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7358a6f-c7e6-4370-9741-10819cceb590 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:20.364054 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1029/2048] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:06:20.395409 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:20.423368 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1020/2049] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:06:20.436175 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7358a6f-c7e6-4370-9741-10819cceb590 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:20.437822 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f7358a6f-c7e6-4370-9741-10819cceb590 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:20.506728 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c44567-6553-42e7-b88a-d4c9a55d5cd2 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:20.508764 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f9c44567-6553-42e7-b88a-d4c9a55d5cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:20.524535 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c44567-6553-42e7-b88a-d4c9a55d5cd2 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:20.529700 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d9f037-0c75-409c-ad2c-968f152f0de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:20.531061 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9f037-0c75-409c-ad2c-968f152f0de4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:20.531061 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9f037-0c75-409c-ad2c-968f152f0de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:20.531061 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9f037-0c75-409c-ad2c-968f152f0de4 admin admin] PATH_INFO: `/v3/users/a5ecdcb3f7844718aea399b360e9e32f` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:20.532651 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d9f037-0c75-409c-ad2c-968f152f0de4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5ecdcb3f7844718aea399b360e9e32f)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:20.534680 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d9f037-0c75-409c-ad2c-968f152f0de4 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:20.546208 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7358a6f-c7e6-4370-9741-10819cceb590 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:20.605037 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563bb8c-d674-4d7a-97ed-d61d08daf15f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:20.605706 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563bb8c-d674-4d7a-97ed-d61d08daf15f service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:20.610957 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563bb8c-d674-4d7a-97ed-d61d08daf15f service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:20.610957 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563bb8c-d674-4d7a-97ed-d61d08daf15f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:20.619004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563bb8c-d674-4d7a-97ed-d61d08daf15f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:20.621175 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563bb8c-d674-4d7a-97ed-d61d08daf15f service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:20.632390 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1030/2050] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:06:20.827204 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-64d9f037-0c75-409c-ad2c-968f152f0de4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5ecdcb3f7844718aea399b360e9e32f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:20.828244 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-64d9f037-0c75-409c-ad2c-968f152f0de4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5ecdcb3f7844718aea399b360e9e32f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:20.836826 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-64d9f037-0c75-409c-ad2c-968f152f0de4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5ecdcb3f7844718aea399b360e9e32f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:20.850358 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-64d9f037-0c75-409c-ad2c-968f152f0de4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5ecdcb3f7844718aea399b360e9e32f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:20.851411 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-64d9f037-0c75-409c-ad2c-968f152f0de4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5ecdcb3f7844718aea399b360e9e32f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:20.968783 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1021/2051] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:06:20 2022] DELETE /identity/v3/users/a5ecdcb3f7844718aea399b360e9e32f => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:20.973965 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23954f4-3a6b-48e7-af77-0e3813705976 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:20.975245 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23954f4-3a6b-48e7-af77-0e3813705976 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:20.975780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23954f4-3a6b-48e7-af77-0e3813705976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:21.000817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c23954f4-3a6b-48e7-af77-0e3813705976 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:21.002165 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c23954f4-3a6b-48e7-af77-0e3813705976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:21.060973 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1031/2052] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:21.071352 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d9f037-0c75-409c-ad2c-968f152f0de4 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:21.072792 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-64d9f037-0c75-409c-ad2c-968f152f0de4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:21.128685 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d9f037-0c75-409c-ad2c-968f152f0de4 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:21.134752 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7328c3-764b-48bf-a5c3-17920f288b81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:21.135372 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7328c3-764b-48bf-a5c3-17920f288b81 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:21.137609 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7328c3-764b-48bf-a5c3-17920f288b81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:21.137609 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7328c3-764b-48bf-a5c3-17920f288b81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:21.145408 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7328c3-764b-48bf-a5c3-17920f288b81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:21.146894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7328c3-764b-48bf-a5c3-17920f288b81 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:21.157124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1022/2053] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:06:21.245579 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:21.790562 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:21.804042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23954f4-3a6b-48e7-af77-0e3813705976 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:21.805902 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c23954f4-3a6b-48e7-af77-0e3813705976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:21.819996 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23954f4-3a6b-48e7-af77-0e3813705976 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:21.834642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c97964-566f-450d-aad5-8f46520c9c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:21.834642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c97964-566f-450d-aad5-8f46520c9c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:21.834642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c97964-566f-450d-aad5-8f46520c9c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:21.834642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c97964-566f-450d-aad5-8f46520c9c8b admin admin] PATH_INFO: `/v3/projects/e56b485033634c68a287c68f41c93a9a` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:21.835467 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c97964-566f-450d-aad5-8f46520c9c8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e56b485033634c68a287c68f41c93a9a)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:21.838133 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c97964-566f-450d-aad5-8f46520c9c8b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:21.994758 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b9c97964-566f-450d-aad5-8f46520c9c8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e56b485033634c68a287c68f41c93a9a', 'request_id': 'req-b9c97964-566f-450d-aad5-8f46520c9c8b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:22.002318 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b9c97964-566f-450d-aad5-8f46520c9c8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e56b485033634c68a287c68f41c93a9a', 'request_id': 'req-b9c97964-566f-450d-aad5-8f46520c9c8b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:22.113493 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b9c97964-566f-450d-aad5-8f46520c9c8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e56b485033634c68a287c68f41c93a9a', 'request_id': 'req-b9c97964-566f-450d-aad5-8f46520c9c8b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:22.124375 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b9c97964-566f-450d-aad5-8f46520c9c8b admin admin] The token cache is being invalidate because project e56b485033634c68a287c68f41c93a9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:06:22.124665 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b9c97964-566f-450d-aad5-8f46520c9c8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:22.125299 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b9c97964-566f-450d-aad5-8f46520c9c8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:22.126774 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1032/2054] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:06:21 2022] DELETE /identity/v3/projects/e56b485033634c68a287c68f41c93a9a => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:22.131539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7328c3-764b-48bf-a5c3-17920f288b81 service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:22.141402 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7328c3-764b-48bf-a5c3-17920f288b81 service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:22.198353 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e34fb9-d85e-4e1a-af1c-ac7cdc2141a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:22.198980 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e34fb9-d85e-4e1a-af1c-ac7cdc2141a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:22.199260 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e34fb9-d85e-4e1a-af1c-ac7cdc2141a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:22.200102 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e34fb9-d85e-4e1a-af1c-ac7cdc2141a5 admin admin] PATH_INFO: `/v3/users/e9685b79f0bc4c64b06ba55bad670b79` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:22.200688 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e34fb9-d85e-4e1a-af1c-ac7cdc2141a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9685b79f0bc4c64b06ba55bad670b79)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:22.202868 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e34fb9-d85e-4e1a-af1c-ac7cdc2141a5 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:22.470530 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-73e34fb9-d85e-4e1a-af1c-ac7cdc2141a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9685b79f0bc4c64b06ba55bad670b79'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:22.471283 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-73e34fb9-d85e-4e1a-af1c-ac7cdc2141a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9685b79f0bc4c64b06ba55bad670b79'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:22.480208 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-73e34fb9-d85e-4e1a-af1c-ac7cdc2141a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9685b79f0bc4c64b06ba55bad670b79'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:22.499294 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-73e34fb9-d85e-4e1a-af1c-ac7cdc2141a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9685b79f0bc4c64b06ba55bad670b79'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:22.500336 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-73e34fb9-d85e-4e1a-af1c-ac7cdc2141a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9685b79f0bc4c64b06ba55bad670b79'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:22.606732 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1023/2055] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:06:22 2022] DELETE /identity/v3/users/e9685b79f0bc4c64b06ba55bad670b79 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:23.090887 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c97964-566f-450d-aad5-8f46520c9c8b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:23.093249 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b9c97964-566f-450d-aad5-8f46520c9c8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:23.111687 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c97964-566f-450d-aad5-8f46520c9c8b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:23.206084 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5f2e8-f1c0-4c70-9565-6aefa65d19ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:23.206734 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5f2e8-f1c0-4c70-9565-6aefa65d19ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:23.207016 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5f2e8-f1c0-4c70-9565-6aefa65d19ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:23.207292 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5f2e8-f1c0-4c70-9565-6aefa65d19ab admin admin] PATH_INFO: `/v3/projects/fc0d7968fa32432f94541f49463cb711` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:23.208903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a5f2e8-f1c0-4c70-9565-6aefa65d19ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc0d7968fa32432f94541f49463cb711)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:23.211701 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a5f2e8-f1c0-4c70-9565-6aefa65d19ab admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:23.352016 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d4a5f2e8-f1c0-4c70-9565-6aefa65d19ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc0d7968fa32432f94541f49463cb711', 'request_id': 'req-d4a5f2e8-f1c0-4c70-9565-6aefa65d19ab'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:23.363331 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d4a5f2e8-f1c0-4c70-9565-6aefa65d19ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc0d7968fa32432f94541f49463cb711', 'request_id': 'req-d4a5f2e8-f1c0-4c70-9565-6aefa65d19ab'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:23.439558 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d4a5f2e8-f1c0-4c70-9565-6aefa65d19ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc0d7968fa32432f94541f49463cb711', 'request_id': 'req-d4a5f2e8-f1c0-4c70-9565-6aefa65d19ab'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:23.456368 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d4a5f2e8-f1c0-4c70-9565-6aefa65d19ab admin admin] The token cache is being invalidate because project fc0d7968fa32432f94541f49463cb711 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:06:23.457297 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d4a5f2e8-f1c0-4c70-9565-6aefa65d19ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:23.457870 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d4a5f2e8-f1c0-4c70-9565-6aefa65d19ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:23.464708 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1033/2056] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:06:23 2022] DELETE /identity/v3/projects/fc0d7968fa32432f94541f49463cb711 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:23.465801 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee22ba-05ff-4c1b-905f-3a34af07f1ac None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:23.466211 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee22ba-05ff-4c1b-905f-3a34af07f1ac None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:23.466555 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee22ba-05ff-4c1b-905f-3a34af07f1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:23.495888 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-33ee22ba-05ff-4c1b-905f-3a34af07f1ac None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:23.497967 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-33ee22ba-05ff-4c1b-905f-3a34af07f1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:23.544420 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:23.591666 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26a995-e6a1-46b2-bcb4-e838c9fa61ff None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:23.592284 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26a995-e6a1-46b2-bcb4-e838c9fa61ff None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:23.597669 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26a995-e6a1-46b2-bcb4-e838c9fa61ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:23.600049 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1024/2057] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:06:23.662626 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:23.676004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9f26a995-e6a1-46b2-bcb4-e838c9fa61ff None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:23.678772 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9f26a995-e6a1-46b2-bcb4-e838c9fa61ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:23.689917 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1034/2058] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:23.699497 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee22ba-05ff-4c1b-905f-3a34af07f1ac None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:23.726736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee22ba-05ff-4c1b-905f-3a34af07f1ac None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:23.750571 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f72c5fa-15f2-4745-9db3-014487c957b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:23.751205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72c5fa-15f2-4745-9db3-014487c957b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:23.751650 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72c5fa-15f2-4745-9db3-014487c957b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:23.752106 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72c5fa-15f2-4745-9db3-014487c957b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:23.752943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f72c5fa-15f2-4745-9db3-014487c957b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:06:23.753404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f72c5fa-15f2-4745-9db3-014487c957b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:23.755222 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f72c5fa-15f2-4745-9db3-014487c957b7 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:23.806205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1025/2059] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:06:23.809534 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26a995-e6a1-46b2-bcb4-e838c9fa61ff None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:23.827388 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26a995-e6a1-46b2-bcb4-e838c9fa61ff None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:23.845410 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdcebf8-aa12-460b-9d3e-5d4c12fc35e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:23.850239 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdcebf8-aa12-460b-9d3e-5d4c12fc35e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:23.850239 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdcebf8-aa12-460b-9d3e-5d4c12fc35e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:23.850239 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdcebf8-aa12-460b-9d3e-5d4c12fc35e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:23.850720 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdcebf8-aa12-460b-9d3e-5d4c12fc35e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:23.856104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdcebf8-aa12-460b-9d3e-5d4c12fc35e7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:24.000301 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1035/2060] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:06:23 2022] POST /identity/v3/projects => generated 373 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:24.001799 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f72c5fa-15f2-4745-9db3-014487c957b7 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:24.018142 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f72c5fa-15f2-4745-9db3-014487c957b7 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:24.163286 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da64d5d-4f28-4a12-9859-6917314eff9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:24.164034 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da64d5d-4f28-4a12-9859-6917314eff9c admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:24.165614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da64d5d-4f28-4a12-9859-6917314eff9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:24.166009 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da64d5d-4f28-4a12-9859-6917314eff9c admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:24.168488 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da64d5d-4f28-4a12-9859-6917314eff9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:24.173820 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da64d5d-4f28-4a12-9859-6917314eff9c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:24.310264 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1026/2061] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:06:24 2022] POST /identity/v3/users => generated 287 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:24.314518 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdcebf8-aa12-460b-9d3e-5d4c12fc35e7 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:24.343010 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdcebf8-aa12-460b-9d3e-5d4c12fc35e7 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:24.367932 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e79f4d-7e7c-4d3a-9276-5e018a61ae6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:24.382355 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e79f4d-7e7c-4d3a-9276-5e018a61ae6c admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:24.382640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e79f4d-7e7c-4d3a-9276-5e018a61ae6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:24.382929 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e79f4d-7e7c-4d3a-9276-5e018a61ae6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:24.405293 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e79f4d-7e7c-4d3a-9276-5e018a61ae6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:24.406982 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e79f4d-7e7c-4d3a-9276-5e018a61ae6c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:24.411461 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da64d5d-4f28-4a12-9859-6917314eff9c admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:24.413953 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8da64d5d-4f28-4a12-9859-6917314eff9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:24.428719 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da64d5d-4f28-4a12-9859-6917314eff9c admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:24.435315 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a985f-e9bd-4d73-bd8d-77f2b3458a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:24.436049 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a985f-e9bd-4d73-bd8d-77f2b3458a21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:24.436575 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a985f-e9bd-4d73-bd8d-77f2b3458a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:24.436777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a985f-e9bd-4d73-bd8d-77f2b3458a21 admin admin] PATH_INFO: `/v3/users/9f56bf91efd54c74a9b4a358f2f9417e` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:24.438184 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a985f-e9bd-4d73-bd8d-77f2b3458a21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f56bf91efd54c74a9b4a358f2f9417e)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:24.440760 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a985f-e9bd-4d73-bd8d-77f2b3458a21 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:24.458676 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1036/2062] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:06:24.463688 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e79f4d-7e7c-4d3a-9276-5e018a61ae6c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:24.492131 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e79f4d-7e7c-4d3a-9276-5e018a61ae6c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:24.501975 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27acee-25d8-4b68-92a2-8d245deef18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:24.504530 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27acee-25d8-4b68-92a2-8d245deef18d admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:24.504926 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27acee-25d8-4b68-92a2-8d245deef18d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:24.505302 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27acee-25d8-4b68-92a2-8d245deef18d admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:24.506131 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d27acee-25d8-4b68-92a2-8d245deef18d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:24.509166 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d27acee-25d8-4b68-92a2-8d245deef18d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:24.523552 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1037/2063] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:06:24.524626 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27acee-25d8-4b68-92a2-8d245deef18d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:24.535285 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27acee-25d8-4b68-92a2-8d245deef18d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:24.541021 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9b027-7f63-464e-b4b0-d4b3f707cf15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:24.554264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9b027-7f63-464e-b4b0-d4b3f707cf15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:24.554553 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9b027-7f63-464e-b4b0-d4b3f707cf15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:24.557174 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9b027-7f63-464e-b4b0-d4b3f707cf15 admin admin] PATH_INFO: `/v3/projects/6afbb47b06b34995919127c7360f0cb6/users/b648149001dd42d4b33b83538114a3ff/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:24.575377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f9b027-7f63-464e-b4b0-d4b3f707cf15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6afbb47b06b34995919127c7360f0cb6, user_id=b648149001dd42d4b33b83538114a3ff, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:24.578436 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f9b027-7f63-464e-b4b0-d4b3f707cf15 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:24.721611 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1038/2064] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:24 2022] PUT /identity/v3/projects/6afbb47b06b34995919127c7360f0cb6/users/b648149001dd42d4b33b83538114a3ff/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:24.722454 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c4a09-5790-4f8f-9e47-cc58a1e6a0bc None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:24.722849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c4a09-5790-4f8f-9e47-cc58a1e6a0bc None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:24.723202 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c4a09-5790-4f8f-9e47-cc58a1e6a0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:24.766255 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f89a985f-e9bd-4d73-bd8d-77f2b3458a21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f56bf91efd54c74a9b4a358f2f9417e'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:24.769073 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f89a985f-e9bd-4d73-bd8d-77f2b3458a21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f56bf91efd54c74a9b4a358f2f9417e'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:24.792267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f89a985f-e9bd-4d73-bd8d-77f2b3458a21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f56bf91efd54c74a9b4a358f2f9417e'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:24.820605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f89a985f-e9bd-4d73-bd8d-77f2b3458a21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f56bf91efd54c74a9b4a358f2f9417e'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:24.822263 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f89a985f-e9bd-4d73-bd8d-77f2b3458a21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f56bf91efd54c74a9b4a358f2f9417e'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:24.845858 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-683c4a09-5790-4f8f-9e47-cc58a1e6a0bc None None] MFA Rules not processed for user `b648149001dd42d4b33b83538114a3ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:24.847145 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-683c4a09-5790-4f8f-9e47-cc58a1e6a0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:24.932415 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1027/2065] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:06:24 2022] DELETE /identity/v3/users/9f56bf91efd54c74a9b4a358f2f9417e => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:24.936744 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4fde10-ab88-4395-bff9-cc5204288dc2 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:24.937035 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4fde10-ab88-4395-bff9-cc5204288dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:24.937332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4fde10-ab88-4395-bff9-cc5204288dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:24.970541 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-9d4fde10-ab88-4395-bff9-cc5204288dc2 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:24.972345 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-9d4fde10-ab88-4395-bff9-cc5204288dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:25.015640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1039/2066] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:06:25.020105 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c4a09-5790-4f8f-9e47-cc58a1e6a0bc None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:25.023678 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-683c4a09-5790-4f8f-9e47-cc58a1e6a0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:25.058084 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1028/2067] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:25.125957 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4fde10-ab88-4395-bff9-cc5204288dc2 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:25.127647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-9d4fde10-ab88-4395-bff9-cc5204288dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:25.166679 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c4a09-5790-4f8f-9e47-cc58a1e6a0bc None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:25.167495 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59af793-1097-49c3-b505-2cb95f81628f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:25.168524 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59af793-1097-49c3-b505-2cb95f81628f admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:25.169084 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59af793-1097-49c3-b505-2cb95f81628f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:25.170080 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59af793-1097-49c3-b505-2cb95f81628f admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:25.170918 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59af793-1097-49c3-b505-2cb95f81628f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:25.173383 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59af793-1097-49c3-b505-2cb95f81628f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:25.240492 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4fde10-ab88-4395-bff9-cc5204288dc2 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:25.245122 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ef8ae-5bf0-4963-ac6c-706e424b3c23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:25.245490 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ef8ae-5bf0-4963-ac6c-706e424b3c23 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:25.245784 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ef8ae-5bf0-4963-ac6c-706e424b3c23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:25.246064 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ef8ae-5bf0-4963-ac6c-706e424b3c23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:25.262817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ef8ae-5bf0-4963-ac6c-706e424b3c23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:25.264456 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ef8ae-5bf0-4963-ac6c-706e424b3c23 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:25.277159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1029/2068] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:06:25.296966 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1040/2069] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:06:25 2022] POST /identity/v3/projects => generated 375 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:25.306628 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ef8ae-5bf0-4963-ac6c-706e424b3c23 service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:25.322196 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ef8ae-5bf0-4963-ac6c-706e424b3c23 service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:25.472841 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189a1c2-78cd-49d4-92fa-13ba6ca920fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:25.472841 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189a1c2-78cd-49d4-92fa-13ba6ca920fc admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:25.472841 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189a1c2-78cd-49d4-92fa-13ba6ca920fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:25.472841 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189a1c2-78cd-49d4-92fa-13ba6ca920fc admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:25.488460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189a1c2-78cd-49d4-92fa-13ba6ca920fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:25.488460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189a1c2-78cd-49d4-92fa-13ba6ca920fc admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:25.654913 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1030/2070] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:06:25 2022] POST /identity/v3/users => generated 294 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:25.657973 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59af793-1097-49c3-b505-2cb95f81628f admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:25.677915 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59af793-1097-49c3-b505-2cb95f81628f admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:25.689853 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71002645-5385-4bc7-be9a-a1a92402776e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:25.690587 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71002645-5385-4bc7-be9a-a1a92402776e admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:25.691327 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71002645-5385-4bc7-be9a-a1a92402776e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:25.691717 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71002645-5385-4bc7-be9a-a1a92402776e admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:25.692533 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71002645-5385-4bc7-be9a-a1a92402776e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:25.695480 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71002645-5385-4bc7-be9a-a1a92402776e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:25.700063 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:25.721827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1041/2071] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:25.725686 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189a1c2-78cd-49d4-92fa-13ba6ca920fc admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:25.736825 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189a1c2-78cd-49d4-92fa-13ba6ca920fc admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:25.742125 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863fc1aa-c242-4b9f-b753-92277e203eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:25.743105 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fc1aa-c242-4b9f-b753-92277e203eaf admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:25.743567 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fc1aa-c242-4b9f-b753-92277e203eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:25.743870 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fc1aa-c242-4b9f-b753-92277e203eaf admin admin] PATH_INFO: `/v3/projects/2c7a56725a124d4da713e3104d596acb/users/cd2df31f09df4405b96f44d8ad189457/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:25.763602 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863fc1aa-c242-4b9f-b753-92277e203eaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c7a56725a124d4da713e3104d596acb, user_id=cd2df31f09df4405b96f44d8ad189457, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:25.766509 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863fc1aa-c242-4b9f-b753-92277e203eaf admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:25.894230 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1031/2072] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:25 2022] PUT /identity/v3/projects/2c7a56725a124d4da713e3104d596acb/users/cd2df31f09df4405b96f44d8ad189457/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:25.899123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71002645-5385-4bc7-be9a-a1a92402776e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:25.912375 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71002645-5385-4bc7-be9a-a1a92402776e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:26.003383 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56fea53-ca1a-4a96-84cd-a859e564152d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:26.004315 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56fea53-ca1a-4a96-84cd-a859e564152d admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:26.004762 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56fea53-ca1a-4a96-84cd-a859e564152d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:26.005385 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56fea53-ca1a-4a96-84cd-a859e564152d admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:26.008352 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56fea53-ca1a-4a96-84cd-a859e564152d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:26.012961 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56fea53-ca1a-4a96-84cd-a859e564152d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:26.023340 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863fc1aa-c242-4b9f-b753-92277e203eaf admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:26.025224 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-863fc1aa-c242-4b9f-b753-92277e203eaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:26.035436 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1042/2073] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:26.040159 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56fea53-ca1a-4a96-84cd-a859e564152d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:26.049307 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863fc1aa-c242-4b9f-b753-92277e203eaf admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:26.057194 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b0aa7-d03c-454e-bfdf-4b2ad1ab43ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:26.060719 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b0aa7-d03c-454e-bfdf-4b2ad1ab43ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:26.060719 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b0aa7-d03c-454e-bfdf-4b2ad1ab43ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:26.060719 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b0aa7-d03c-454e-bfdf-4b2ad1ab43ab admin admin] PATH_INFO: `/v3/projects/951eca8992d347be9672b9f3a27741af` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:26.061692 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b0aa7-d03c-454e-bfdf-4b2ad1ab43ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=951eca8992d347be9672b9f3a27741af)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:26.063238 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56fea53-ca1a-4a96-84cd-a859e564152d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:26.067158 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b0aa7-d03c-454e-bfdf-4b2ad1ab43ab admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:26.081061 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd214c80-8ac9-4aed-8077-d465a389cf4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:26.081769 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd214c80-8ac9-4aed-8077-d465a389cf4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:26.082054 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd214c80-8ac9-4aed-8077-d465a389cf4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:26.082330 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd214c80-8ac9-4aed-8077-d465a389cf4e admin admin] PATH_INFO: `/v3/projects/2c7a56725a124d4da713e3104d596acb/users/cd2df31f09df4405b96f44d8ad189457/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:26.085713 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd214c80-8ac9-4aed-8077-d465a389cf4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c7a56725a124d4da713e3104d596acb, user_id=cd2df31f09df4405b96f44d8ad189457, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:26.091484 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd214c80-8ac9-4aed-8077-d465a389cf4e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:26.111972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1043/2074] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:26 2022] PUT /identity/v3/projects/2c7a56725a124d4da713e3104d596acb/users/cd2df31f09df4405b96f44d8ad189457/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:26.124894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a890e-5f04-4fcb-8bdf-cca91be1d997 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:26.125938 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a890e-5f04-4fcb-8bdf-cca91be1d997 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:26.126306 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a890e-5f04-4fcb-8bdf-cca91be1d997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:26.164109 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5b1a890e-5f04-4fcb-8bdf-cca91be1d997 None None] MFA Rules not processed for user `cd2df31f09df4405b96f44d8ad189457`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:26.164963 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5b1a890e-5f04-4fcb-8bdf-cca91be1d997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:26.269485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1044/2075] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:26.269795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-186b0aa7-d03c-454e-bfdf-4b2ad1ab43ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '951eca8992d347be9672b9f3a27741af', 'request_id': 'req-186b0aa7-d03c-454e-bfdf-4b2ad1ab43ab'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:26.282012 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a890e-5f04-4fcb-8bdf-cca91be1d997 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:26.289196 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5b1a890e-5f04-4fcb-8bdf-cca91be1d997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:26.342665 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:26.371929 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-186b0aa7-d03c-454e-bfdf-4b2ad1ab43ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '951eca8992d347be9672b9f3a27741af', 'request_id': 'req-186b0aa7-d03c-454e-bfdf-4b2ad1ab43ab'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:26.384494 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-186b0aa7-d03c-454e-bfdf-4b2ad1ab43ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '951eca8992d347be9672b9f3a27741af', 'request_id': 'req-186b0aa7-d03c-454e-bfdf-4b2ad1ab43ab'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:26.407593 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-186b0aa7-d03c-454e-bfdf-4b2ad1ab43ab admin admin] The token cache is being invalidate because project 951eca8992d347be9672b9f3a27741af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:06:26.410400 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-186b0aa7-d03c-454e-bfdf-4b2ad1ab43ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:26.413504 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-186b0aa7-d03c-454e-bfdf-4b2ad1ab43ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:26.415797 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1032/2076] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:06:26 2022] DELETE /identity/v3/projects/951eca8992d347be9672b9f3a27741af => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:06:26.418661 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a890e-5f04-4fcb-8bdf-cca91be1d997 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:26.420535 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b0aa7-d03c-454e-bfdf-4b2ad1ab43ab admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:26.422895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-186b0aa7-d03c-454e-bfdf-4b2ad1ab43ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:26.460007 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:26.509269 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca786fd8-4c2d-4307-9748-f00fab88436b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:26.510243 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca786fd8-4c2d-4307-9748-f00fab88436b service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:26.510651 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca786fd8-4c2d-4307-9748-f00fab88436b service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:26.511199 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca786fd8-4c2d-4307-9748-f00fab88436b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:26.517843 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ca786fd8-4c2d-4307-9748-f00fab88436b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:26.583792 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b0aa7-d03c-454e-bfdf-4b2ad1ab43ab admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:26.595203 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e862ef1b-ebc9-43ef-8a5b-1c7f80a66131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:26.598862 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862ef1b-ebc9-43ef-8a5b-1c7f80a66131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:26.599357 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862ef1b-ebc9-43ef-8a5b-1c7f80a66131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:26.605391 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862ef1b-ebc9-43ef-8a5b-1c7f80a66131 admin admin] PATH_INFO: `/v3/users/f9b51edde0a543aaadc7933a8e5cc0d3` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:26.605391 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e862ef1b-ebc9-43ef-8a5b-1c7f80a66131 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9b51edde0a543aaadc7933a8e5cc0d3)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:26.605949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e862ef1b-ebc9-43ef-8a5b-1c7f80a66131 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:26.655378 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca786fd8-4c2d-4307-9748-f00fab88436b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:26.657222 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca786fd8-4c2d-4307-9748-f00fab88436b service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:26.670587 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1045/2077] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:06:26.821135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e862ef1b-ebc9-43ef-8a5b-1c7f80a66131 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9b51edde0a543aaadc7933a8e5cc0d3'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:26.822586 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e862ef1b-ebc9-43ef-8a5b-1c7f80a66131 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9b51edde0a543aaadc7933a8e5cc0d3'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:26.842892 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e862ef1b-ebc9-43ef-8a5b-1c7f80a66131 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9b51edde0a543aaadc7933a8e5cc0d3'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:26.872789 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e862ef1b-ebc9-43ef-8a5b-1c7f80a66131 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9b51edde0a543aaadc7933a8e5cc0d3'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:26.873547 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e862ef1b-ebc9-43ef-8a5b-1c7f80a66131 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9b51edde0a543aaadc7933a8e5cc0d3'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:27.006523 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1033/2078] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:06:26 2022] DELETE /identity/v3/users/f9b51edde0a543aaadc7933a8e5cc0d3 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:27.594988 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca786fd8-4c2d-4307-9748-f00fab88436b service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:27.610274 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca786fd8-4c2d-4307-9748-f00fab88436b service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:27.691939 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba183b46-9a2a-4dc6-b2fd-bf2dd1e914b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:27.692883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba183b46-9a2a-4dc6-b2fd-bf2dd1e914b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:27.693270 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba183b46-9a2a-4dc6-b2fd-bf2dd1e914b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:27.693721 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba183b46-9a2a-4dc6-b2fd-bf2dd1e914b4 admin admin] PATH_INFO: `/v3/projects/ec7c11e1f3004151b8922799abff7eb5` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:27.695428 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba183b46-9a2a-4dc6-b2fd-bf2dd1e914b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec7c11e1f3004151b8922799abff7eb5)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:27.698767 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba183b46-9a2a-4dc6-b2fd-bf2dd1e914b4 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:27.842594 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ba183b46-9a2a-4dc6-b2fd-bf2dd1e914b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec7c11e1f3004151b8922799abff7eb5', 'request_id': 'req-ba183b46-9a2a-4dc6-b2fd-bf2dd1e914b4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:27.851996 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ba183b46-9a2a-4dc6-b2fd-bf2dd1e914b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec7c11e1f3004151b8922799abff7eb5', 'request_id': 'req-ba183b46-9a2a-4dc6-b2fd-bf2dd1e914b4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:27.974155 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ba183b46-9a2a-4dc6-b2fd-bf2dd1e914b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec7c11e1f3004151b8922799abff7eb5', 'request_id': 'req-ba183b46-9a2a-4dc6-b2fd-bf2dd1e914b4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:27.990785 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ba183b46-9a2a-4dc6-b2fd-bf2dd1e914b4 admin admin] The token cache is being invalidate because project ec7c11e1f3004151b8922799abff7eb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:06:27.994225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ba183b46-9a2a-4dc6-b2fd-bf2dd1e914b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:27.995611 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ba183b46-9a2a-4dc6-b2fd-bf2dd1e914b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:27.998986 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1046/2079] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:06:27 2022] DELETE /identity/v3/projects/ec7c11e1f3004151b8922799abff7eb5 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:28.005307 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c3b45-9beb-40a9-b74f-7d02ff603859 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:28.014009 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c3b45-9beb-40a9-b74f-7d02ff603859 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:28.023874 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c3b45-9beb-40a9-b74f-7d02ff603859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:28.050816 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-ef8c3b45-9beb-40a9-b74f-7d02ff603859 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:28.052900 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ef8c3b45-9beb-40a9-b74f-7d02ff603859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:28.174831 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1034/2080] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:06:28.179731 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0381f3-506b-4018-8a58-b9bba1b58fe0 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:28.180025 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0381f3-506b-4018-8a58-b9bba1b58fe0 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:28.180340 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0381f3-506b-4018-8a58-b9bba1b58fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:28.213585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-fc0381f3-506b-4018-8a58-b9bba1b58fe0 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:28.215443 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fc0381f3-506b-4018-8a58-b9bba1b58fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:28.234150 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1047/2081] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:28.238100 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c3b45-9beb-40a9-b74f-7d02ff603859 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:28.250434 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c3b45-9beb-40a9-b74f-7d02ff603859 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:28.257960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea4faf-421c-43e4-9d74-4adc3ab45c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:28.260827 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea4faf-421c-43e4-9d74-4adc3ab45c96 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:28.260827 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea4faf-421c-43e4-9d74-4adc3ab45c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:28.260827 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea4faf-421c-43e4-9d74-4adc3ab45c96 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:28.260827 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea4faf-421c-43e4-9d74-4adc3ab45c96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:06:28.260827 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea4faf-421c-43e4-9d74-4adc3ab45c96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:28.264362 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea4faf-421c-43e4-9d74-4adc3ab45c96 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:28.279725 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1035/2082] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:06:28.283913 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0381f3-506b-4018-8a58-b9bba1b58fe0 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:28.326628 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0381f3-506b-4018-8a58-b9bba1b58fe0 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:28.337917 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3efa1-f859-4cc5-95f1-d07250200b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:28.338655 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3efa1-f859-4cc5-95f1-d07250200b54 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:28.338997 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3efa1-f859-4cc5-95f1-d07250200b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:28.341242 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3efa1-f859-4cc5-95f1-d07250200b54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:28.341242 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a3efa1-f859-4cc5-95f1-d07250200b54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:28.343672 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a3efa1-f859-4cc5-95f1-d07250200b54 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:28.460129 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1048/2083] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:06:28 2022] POST /identity/v3/projects => generated 371 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:06:28.465462 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea4faf-421c-43e4-9d74-4adc3ab45c96 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:28.486748 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea4faf-421c-43e4-9d74-4adc3ab45c96 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:28.627561 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80063bc9-4a5d-49ea-9a01-40d1bd25d2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:28.628742 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80063bc9-4a5d-49ea-9a01-40d1bd25d2a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:28.634903 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80063bc9-4a5d-49ea-9a01-40d1bd25d2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:28.634903 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80063bc9-4a5d-49ea-9a01-40d1bd25d2a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:28.634903 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80063bc9-4a5d-49ea-9a01-40d1bd25d2a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:28.634903 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80063bc9-4a5d-49ea-9a01-40d1bd25d2a2 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:28.785690 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1036/2084] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:06:28 2022] POST /identity/v3/users => generated 286 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:28.788323 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3efa1-f859-4cc5-95f1-d07250200b54 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:28.802814 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3efa1-f859-4cc5-95f1-d07250200b54 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:28.817723 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d3cbda-2590-4e55-aa1f-67170ddcff02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:28.818904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3cbda-2590-4e55-aa1f-67170ddcff02 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:28.819186 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3cbda-2590-4e55-aa1f-67170ddcff02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:28.819459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3cbda-2590-4e55-aa1f-67170ddcff02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:28.820149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d3cbda-2590-4e55-aa1f-67170ddcff02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:28.821797 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d3cbda-2590-4e55-aa1f-67170ddcff02 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:28.838910 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1049/2085] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:28.843840 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80063bc9-4a5d-49ea-9a01-40d1bd25d2a2 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:28.858185 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80063bc9-4a5d-49ea-9a01-40d1bd25d2a2 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:28.864864 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351bb58-3b78-446a-b8e4-05c823a3e36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:28.865874 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351bb58-3b78-446a-b8e4-05c823a3e36a admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:28.866221 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351bb58-3b78-446a-b8e4-05c823a3e36a admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:28.866572 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351bb58-3b78-446a-b8e4-05c823a3e36a admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:28.867371 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6351bb58-3b78-446a-b8e4-05c823a3e36a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:28.869123 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6351bb58-3b78-446a-b8e4-05c823a3e36a admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:28.883367 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1037/2086] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:28.897030 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d3cbda-2590-4e55-aa1f-67170ddcff02 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:28.899204 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d3cbda-2590-4e55-aa1f-67170ddcff02 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:28.905279 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d76f6-af19-4d4e-a5da-34e4a5df07b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:28.905976 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d76f6-af19-4d4e-a5da-34e4a5df07b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:28.906257 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d76f6-af19-4d4e-a5da-34e4a5df07b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:28.906530 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d76f6-af19-4d4e-a5da-34e4a5df07b6 admin admin] PATH_INFO: `/v3/projects/47af18d14ddd400898b8cf3256af985b/users/b75c1f61554d489cb035e6af6819bd02/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:28.946389 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d76f6-af19-4d4e-a5da-34e4a5df07b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47af18d14ddd400898b8cf3256af985b, user_id=b75c1f61554d489cb035e6af6819bd02, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:28.949143 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d76f6-af19-4d4e-a5da-34e4a5df07b6 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:29.034125 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1050/2087] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:28 2022] PUT /identity/v3/projects/47af18d14ddd400898b8cf3256af985b/users/b75c1f61554d489cb035e6af6819bd02/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:29.039558 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8f845-c94a-4c03-a441-72584c2602c7 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:29.040063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8f845-c94a-4c03-a441-72584c2602c7 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:29.040178 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8f845-c94a-4c03-a441-72584c2602c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:29.061428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-a0f8f845-c94a-4c03-a441-72584c2602c7 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:29.063159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a0f8f845-c94a-4c03-a441-72584c2602c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:29.073935 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be6216-03f9-44b1-b49c-1bc6ee440f01 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:29.074224 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be6216-03f9-44b1-b49c-1bc6ee440f01 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:29.075071 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be6216-03f9-44b1-b49c-1bc6ee440f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:29.105715 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-37be6216-03f9-44b1-b49c-1bc6ee440f01 None None] MFA Rules not processed for user `b17adcb2be25427485d5004c7821e094`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:29.106930 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-37be6216-03f9-44b1-b49c-1bc6ee440f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:29.132688 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1038/2088] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:29.151820 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8f845-c94a-4c03-a441-72584c2602c7 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:29.151820 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a0f8f845-c94a-4c03-a441-72584c2602c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:29.179435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1051/2089] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:06:29.184125 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be6216-03f9-44b1-b49c-1bc6ee440f01 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:29.185480 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-37be6216-03f9-44b1-b49c-1bc6ee440f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:29.199657 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be6216-03f9-44b1-b49c-1bc6ee440f01 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:29.204843 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7dfa82-1997-4c3f-a320-ab0bd8d92cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:29.205401 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7dfa82-1997-4c3f-a320-ab0bd8d92cf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:29.205811 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7dfa82-1997-4c3f-a320-ab0bd8d92cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:29.206083 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7dfa82-1997-4c3f-a320-ab0bd8d92cf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:29.211897 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7dfa82-1997-4c3f-a320-ab0bd8d92cf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:29.212022 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8f845-c94a-4c03-a441-72584c2602c7 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:29.212157 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7dfa82-1997-4c3f-a320-ab0bd8d92cf0 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:29.212256 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40220c58-cb22-4546-b8be-4b371420f1eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:29.212256 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40220c58-cb22-4546-b8be-4b371420f1eb service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:29.212256 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40220c58-cb22-4546-b8be-4b371420f1eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:29.212256 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40220c58-cb22-4546-b8be-4b371420f1eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:29.221403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40220c58-cb22-4546-b8be-4b371420f1eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:29.222827 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40220c58-cb22-4546-b8be-4b371420f1eb service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:29.232833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1039/2090] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:06:29.258512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:29.323031 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1052/2091] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:06:29 2022] POST /identity/v3/projects => generated 371 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:06:29.327404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40220c58-cb22-4546-b8be-4b371420f1eb service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:29.337560 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40220c58-cb22-4546-b8be-4b371420f1eb service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:29.405566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:29.414624 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148969bc-0055-4665-aefd-5de5f8e4f4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:29.416423 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148969bc-0055-4665-aefd-5de5f8e4f4d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:29.416423 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148969bc-0055-4665-aefd-5de5f8e4f4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:29.416423 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148969bc-0055-4665-aefd-5de5f8e4f4d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:29.416963 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148969bc-0055-4665-aefd-5de5f8e4f4d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:29.419092 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148969bc-0055-4665-aefd-5de5f8e4f4d9 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:29.525742 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1040/2092] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:06:29 2022] POST /identity/v3/users => generated 292 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:06:29.529804 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7dfa82-1997-4c3f-a320-ab0bd8d92cf0 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:29.548929 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7dfa82-1997-4c3f-a320-ab0bd8d92cf0 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:29.555209 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2c925-e9d8-4de2-b630-ddf575fb87bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:29.556006 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2c925-e9d8-4de2-b630-ddf575fb87bb admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:29.556496 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2c925-e9d8-4de2-b630-ddf575fb87bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:29.556864 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2c925-e9d8-4de2-b630-ddf575fb87bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:29.557656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b2c925-e9d8-4de2-b630-ddf575fb87bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:29.561246 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b2c925-e9d8-4de2-b630-ddf575fb87bb admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:29.577978 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148969bc-0055-4665-aefd-5de5f8e4f4d9 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:29.580576 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1053/2093] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:29.597157 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148969bc-0055-4665-aefd-5de5f8e4f4d9 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:29.608488 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f374f-a872-4f8a-8879-f6cb04af2afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:29.609309 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f374f-a872-4f8a-8879-f6cb04af2afb admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:29.609599 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f374f-a872-4f8a-8879-f6cb04af2afb admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:29.609877 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f374f-a872-4f8a-8879-f6cb04af2afb admin admin] PATH_INFO: `/v3/projects/c51c747a278c42608b5091d206ff29cc/users/6bcc5313cdc04538a6b0f344cd369aab/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:29.843179 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f374f-a872-4f8a-8879-f6cb04af2afb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c51c747a278c42608b5091d206ff29cc, user_id=6bcc5313cdc04538a6b0f344cd369aab, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:29.843179 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f374f-a872-4f8a-8879-f6cb04af2afb admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:29.852936 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2c925-e9d8-4de2-b630-ddf575fb87bb admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:29.852936 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c9b2c925-e9d8-4de2-b630-ddf575fb87bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:29.852936 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2c925-e9d8-4de2-b630-ddf575fb87bb admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:29.854251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14e1c3-2eff-4f38-aa1f-919a0c3a3143 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:29.854251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14e1c3-2eff-4f38-aa1f-919a0c3a3143 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:29.854251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14e1c3-2eff-4f38-aa1f-919a0c3a3143 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:29.854251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14e1c3-2eff-4f38-aa1f-919a0c3a3143 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:29.857542 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1f14e1c3-2eff-4f38-aa1f-919a0c3a3143 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:29.858064 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1041/2094] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:29 2022] PUT /identity/v3/projects/c51c747a278c42608b5091d206ff29cc/users/6bcc5313cdc04538a6b0f344cd369aab/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:29.858064 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f374f-a872-4f8a-8879-f6cb04af2afb admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:29.931188 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f14e1c3-2eff-4f38-aa1f-919a0c3a3143 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:29.932695 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f14e1c3-2eff-4f38-aa1f-919a0c3a3143 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:29.995830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1054/2095] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:29 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:06:30.006852 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f374f-a872-4f8a-8879-f6cb04af2afb admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:30.030446 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64337683-3410-4a1b-8791-b962a5bda03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:30.031300 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64337683-3410-4a1b-8791-b962a5bda03b admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:30.034976 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64337683-3410-4a1b-8791-b962a5bda03b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:30.035668 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64337683-3410-4a1b-8791-b962a5bda03b admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:30.036508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64337683-3410-4a1b-8791-b962a5bda03b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:30.039732 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64337683-3410-4a1b-8791-b962a5bda03b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:30.070078 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1042/2096] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:06:30.074490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14e1c3-2eff-4f38-aa1f-919a0c3a3143 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:30.091941 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14e1c3-2eff-4f38-aa1f-919a0c3a3143 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:30.105522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb6397-851a-4fec-9e52-64538350db8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:30.106307 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb6397-851a-4fec-9e52-64538350db8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:30.106673 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb6397-851a-4fec-9e52-64538350db8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:30.108285 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb6397-851a-4fec-9e52-64538350db8f admin admin] PATH_INFO: `/v3/projects/c51c747a278c42608b5091d206ff29cc/users/6bcc5313cdc04538a6b0f344cd369aab/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:30.112956 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bb6397-851a-4fec-9e52-64538350db8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c51c747a278c42608b5091d206ff29cc, user_id=6bcc5313cdc04538a6b0f344cd369aab, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:30.115823 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bb6397-851a-4fec-9e52-64538350db8f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:30.130212 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1055/2097] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:30 2022] PUT /identity/v3/projects/c51c747a278c42608b5091d206ff29cc/users/6bcc5313cdc04538a6b0f344cd369aab/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:06:30.443638 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:DELETE=7,keystone:SELECT=87,keystone:INSERT=6 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:31.381104 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64337683-3410-4a1b-8791-b962a5bda03b admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:31.382997 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-64337683-3410-4a1b-8791-b962a5bda03b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:31.400342 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64337683-3410-4a1b-8791-b962a5bda03b admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:31.489896 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c6db4-daed-46ab-ba3c-d5916604620e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:31.490916 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c6db4-daed-46ab-ba3c-d5916604620e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:31.491407 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c6db4-daed-46ab-ba3c-d5916604620e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:31.491752 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c6db4-daed-46ab-ba3c-d5916604620e admin admin] PATH_INFO: `/v3/users/b648149001dd42d4b33b83538114a3ff` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:31.493378 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35c6db4-daed-46ab-ba3c-d5916604620e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b648149001dd42d4b33b83538114a3ff)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:31.495681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35c6db4-daed-46ab-ba3c-d5916604620e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:31.620725 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13635300-2cdc-466a-8e68-0d2bcfd69da9 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:31.621612 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13635300-2cdc-466a-8e68-0d2bcfd69da9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:31.621899 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13635300-2cdc-466a-8e68-0d2bcfd69da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:31.653027 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-13635300-2cdc-466a-8e68-0d2bcfd69da9 None None] MFA Rules not processed for user `b75c1f61554d489cb035e6af6819bd02`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:31.655201 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-13635300-2cdc-466a-8e68-0d2bcfd69da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:31.718223 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-c35c6db4-daed-46ab-ba3c-d5916604620e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b648149001dd42d4b33b83538114a3ff'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:31.718901 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-c35c6db4-daed-46ab-ba3c-d5916604620e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b648149001dd42d4b33b83538114a3ff'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:31.730771 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-c35c6db4-daed-46ab-ba3c-d5916604620e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b648149001dd42d4b33b83538114a3ff'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:31.743931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-c35c6db4-daed-46ab-ba3c-d5916604620e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b648149001dd42d4b33b83538114a3ff'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:31.744800 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-c35c6db4-daed-46ab-ba3c-d5916604620e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b648149001dd42d4b33b83538114a3ff'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:31.800324 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1056/2098] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:06:31.801680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13635300-2cdc-466a-8e68-0d2bcfd69da9 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:31.812137 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-13635300-2cdc-466a-8e68-0d2bcfd69da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:31.840636 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13635300-2cdc-466a-8e68-0d2bcfd69da9 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:31.852582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c27eb-fd5b-4def-9ba0-b56bf33cf15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:31.852582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c27eb-fd5b-4def-9ba0-b56bf33cf15e admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:31.852582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c27eb-fd5b-4def-9ba0-b56bf33cf15e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:31.852582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c27eb-fd5b-4def-9ba0-b56bf33cf15e admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:31.853288 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c27eb-fd5b-4def-9ba0-b56bf33cf15e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:31.853968 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c27eb-fd5b-4def-9ba0-b56bf33cf15e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:31.880034 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1043/2099] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:06:31 2022] DELETE /identity/v3/users/b648149001dd42d4b33b83538114a3ff => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:31.886700 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594196d0-e53c-4364-b857-e8b8c04cb05e None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:31.886700 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594196d0-e53c-4364-b857-e8b8c04cb05e None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:31.886700 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594196d0-e53c-4364-b857-e8b8c04cb05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:31.925979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-594196d0-e53c-4364-b857-e8b8c04cb05e None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:31.927468 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-594196d0-e53c-4364-b857-e8b8c04cb05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:31.975278 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1057/2100] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:06:31 2022] POST /identity/v3/projects => generated 373 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:31.976319 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c27eb-fd5b-4def-9ba0-b56bf33cf15e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:32.024329 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1044/2101] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:06:32.041401 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:32.043494 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594196d0-e53c-4364-b857-e8b8c04cb05e None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:32.045474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-594196d0-e53c-4364-b857-e8b8c04cb05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:32.107936 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c27eb-fd5b-4def-9ba0-b56bf33cf15e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:32.113901 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f7053-349a-48ad-a919-09bca0f55e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:32.114529 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f7053-349a-48ad-a919-09bca0f55e88 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:32.115549 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f7053-349a-48ad-a919-09bca0f55e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:32.115671 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f7053-349a-48ad-a919-09bca0f55e88 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:32.116316 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f7053-349a-48ad-a919-09bca0f55e88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:32.118550 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f7053-349a-48ad-a919-09bca0f55e88 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:32.123255 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594196d0-e53c-4364-b857-e8b8c04cb05e None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:32.221736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af00d23-20e8-40af-b539-562d37dfb2ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:32.222680 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af00d23-20e8-40af-b539-562d37dfb2ef service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:32.225969 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af00d23-20e8-40af-b539-562d37dfb2ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:32.227078 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af00d23-20e8-40af-b539-562d37dfb2ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:32.250386 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af00d23-20e8-40af-b539-562d37dfb2ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:32.252033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af00d23-20e8-40af-b539-562d37dfb2ef service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:32.272116 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1058/2102] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:06:31 2022] POST /identity/v3/users => generated 293 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:32.273345 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f7053-349a-48ad-a919-09bca0f55e88 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:32.278361 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1045/2103] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:06:32.284572 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f7053-349a-48ad-a919-09bca0f55e88 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:32.303965 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87142fb-f79c-4c70-ad97-2a3a56987118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:32.304723 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87142fb-f79c-4c70-ad97-2a3a56987118 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:32.305379 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87142fb-f79c-4c70-ad97-2a3a56987118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:32.306146 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87142fb-f79c-4c70-ad97-2a3a56987118 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:32.308129 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87142fb-f79c-4c70-ad97-2a3a56987118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:32.310586 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87142fb-f79c-4c70-ad97-2a3a56987118 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:32.328726 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1059/2104] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:32.330125 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af00d23-20e8-40af-b539-562d37dfb2ef service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:32.341815 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af00d23-20e8-40af-b539-562d37dfb2ef service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:32.348952 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a38c3e-d7b0-4ab5-8b64-6055b667b81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:32.349363 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a38c3e-d7b0-4ab5-8b64-6055b667b81c admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:32.349645 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a38c3e-d7b0-4ab5-8b64-6055b667b81c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:32.349923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a38c3e-d7b0-4ab5-8b64-6055b667b81c admin admin] PATH_INFO: `/v3/projects/6e48e202b622484ba250047d3a33c091/users/f950ed50da104a469cb3efa7bc3ab6f9/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:32.386011 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a38c3e-d7b0-4ab5-8b64-6055b667b81c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e48e202b622484ba250047d3a33c091, user_id=f950ed50da104a469cb3efa7bc3ab6f9, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:32.390163 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a38c3e-d7b0-4ab5-8b64-6055b667b81c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:32.505857 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1046/2105] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:32 2022] PUT /identity/v3/projects/6e48e202b622484ba250047d3a33c091/users/f950ed50da104a469cb3efa7bc3ab6f9/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:32.509907 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87142fb-f79c-4c70-ad97-2a3a56987118 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:32.524496 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87142fb-f79c-4c70-ad97-2a3a56987118 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:32.621986 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba16d5-3a50-4c3c-9d0b-e4142f2461b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:32.622732 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba16d5-3a50-4c3c-9d0b-e4142f2461b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:32.623147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba16d5-3a50-4c3c-9d0b-e4142f2461b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:32.623565 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba16d5-3a50-4c3c-9d0b-e4142f2461b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:32.624383 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba16d5-3a50-4c3c-9d0b-e4142f2461b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:32.626470 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba16d5-3a50-4c3c-9d0b-e4142f2461b9 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:32.644279 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a38c3e-d7b0-4ab5-8b64-6055b667b81c admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:32.651363 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1060/2106] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:32.660403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a38c3e-d7b0-4ab5-8b64-6055b667b81c admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:32.668393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67805360-2e44-4d6f-afec-df72d80f549a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:32.674697 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67805360-2e44-4d6f-afec-df72d80f549a admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:32.674697 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67805360-2e44-4d6f-afec-df72d80f549a admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:32.674697 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67805360-2e44-4d6f-afec-df72d80f549a admin admin] PATH_INFO: `/v3/projects/6e48e202b622484ba250047d3a33c091/users/f950ed50da104a469cb3efa7bc3ab6f9/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:32.674697 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67805360-2e44-4d6f-afec-df72d80f549a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e48e202b622484ba250047d3a33c091, user_id=f950ed50da104a469cb3efa7bc3ab6f9, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:32.678270 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67805360-2e44-4d6f-afec-df72d80f549a admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:32.697623 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1047/2107] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:32 2022] PUT /identity/v3/projects/6e48e202b622484ba250047d3a33c091/users/f950ed50da104a469cb3efa7bc3ab6f9/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:06:33.089658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:33.385788 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba16d5-3a50-4c3c-9d0b-e4142f2461b9 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:33.389267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-00ba16d5-3a50-4c3c-9d0b-e4142f2461b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:33.425616 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba16d5-3a50-4c3c-9d0b-e4142f2461b9 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:33.562950 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf1517a-655b-4027-a4f4-87649a46817f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:33.565125 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1517a-655b-4027-a4f4-87649a46817f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:33.565411 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1517a-655b-4027-a4f4-87649a46817f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:33.565689 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1517a-655b-4027-a4f4-87649a46817f admin admin] PATH_INFO: `/v3/projects/6afbb47b06b34995919127c7360f0cb6` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:33.569991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf1517a-655b-4027-a4f4-87649a46817f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6afbb47b06b34995919127c7360f0cb6)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:33.574399 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf1517a-655b-4027-a4f4-87649a46817f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:33.756998 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-3bf1517a-655b-4027-a4f4-87649a46817f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6afbb47b06b34995919127c7360f0cb6', 'request_id': 'req-3bf1517a-655b-4027-a4f4-87649a46817f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:33.774516 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-3bf1517a-655b-4027-a4f4-87649a46817f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6afbb47b06b34995919127c7360f0cb6', 'request_id': 'req-3bf1517a-655b-4027-a4f4-87649a46817f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:33.945675 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-3bf1517a-655b-4027-a4f4-87649a46817f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6afbb47b06b34995919127c7360f0cb6', 'request_id': 'req-3bf1517a-655b-4027-a4f4-87649a46817f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:33.981036 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-3bf1517a-655b-4027-a4f4-87649a46817f admin admin] The token cache is being invalidate because project 6afbb47b06b34995919127c7360f0cb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:06:33.982452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-3bf1517a-655b-4027-a4f4-87649a46817f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:33.983424 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-3bf1517a-655b-4027-a4f4-87649a46817f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:33.989563 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1061/2108] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:06:33 2022] DELETE /identity/v3/projects/6afbb47b06b34995919127c7360f0cb6 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:33.994013 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67805360-2e44-4d6f-afec-df72d80f549a admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:33.999299 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-67805360-2e44-4d6f-afec-df72d80f549a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:34.024528 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67805360-2e44-4d6f-afec-df72d80f549a admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:34.157974 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da3b42-b3d7-403f-9835-7579830acc52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:34.158649 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da3b42-b3d7-403f-9835-7579830acc52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:34.159605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da3b42-b3d7-403f-9835-7579830acc52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:34.159908 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da3b42-b3d7-403f-9835-7579830acc52 admin admin] PATH_INFO: `/v3/users/cd2df31f09df4405b96f44d8ad189457` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:34.161504 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7da3b42-b3d7-403f-9835-7579830acc52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd2df31f09df4405b96f44d8ad189457)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:34.164281 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7da3b42-b3d7-403f-9835-7579830acc52 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:34.444999 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b7da3b42-b3d7-403f-9835-7579830acc52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd2df31f09df4405b96f44d8ad189457'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:34.445716 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b7da3b42-b3d7-403f-9835-7579830acc52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd2df31f09df4405b96f44d8ad189457'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:34.472598 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b7da3b42-b3d7-403f-9835-7579830acc52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd2df31f09df4405b96f44d8ad189457'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:34.497177 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b7da3b42-b3d7-403f-9835-7579830acc52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd2df31f09df4405b96f44d8ad189457'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:34.497935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b7da3b42-b3d7-403f-9835-7579830acc52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd2df31f09df4405b96f44d8ad189457'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:34.602303 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1048/2109] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:06:33 2022] DELETE /identity/v3/users/cd2df31f09df4405b96f44d8ad189457 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:34.910523 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:35.113950 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:35.341772 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf1517a-655b-4027-a4f4-87649a46817f admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:35.357231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf1517a-655b-4027-a4f4-87649a46817f admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:35.415469 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7812c4-4e8a-4773-a297-3e553e44832d None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:35.416519 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7812c4-4e8a-4773-a297-3e553e44832d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:35.417055 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7812c4-4e8a-4773-a297-3e553e44832d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:35.479685 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-de7812c4-4e8a-4773-a297-3e553e44832d None None] MFA Rules not processed for user `f950ed50da104a469cb3efa7bc3ab6f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:35.480972 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-de7812c4-4e8a-4773-a297-3e553e44832d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:35.489170 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6079b-8dd8-4ed0-b306-62ea4a1de6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:35.489326 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6079b-8dd8-4ed0-b306-62ea4a1de6fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:35.489548 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6079b-8dd8-4ed0-b306-62ea4a1de6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:35.489852 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6079b-8dd8-4ed0-b306-62ea4a1de6fc admin admin] PATH_INFO: `/v3/projects/2c7a56725a124d4da713e3104d596acb` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:35.493792 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a6079b-8dd8-4ed0-b306-62ea4a1de6fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c7a56725a124d4da713e3104d596acb)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:35.496793 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a6079b-8dd8-4ed0-b306-62ea4a1de6fc admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:35.600463 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1049/2110] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:35 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:35.620348 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7812c4-4e8a-4773-a297-3e553e44832d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:35.622054 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-de7812c4-4e8a-4773-a297-3e553e44832d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:35.692276 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-58a6079b-8dd8-4ed0-b306-62ea4a1de6fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c7a56725a124d4da713e3104d596acb', 'request_id': 'req-58a6079b-8dd8-4ed0-b306-62ea4a1de6fc'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:35.703841 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-58a6079b-8dd8-4ed0-b306-62ea4a1de6fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c7a56725a124d4da713e3104d596acb', 'request_id': 'req-58a6079b-8dd8-4ed0-b306-62ea4a1de6fc'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:35.719116 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7812c4-4e8a-4773-a297-3e553e44832d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:35.789467 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607b8ab-b969-4c8c-8950-5550f2bcff12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:35.790142 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607b8ab-b969-4c8c-8950-5550f2bcff12 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:35.790423 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607b8ab-b969-4c8c-8950-5550f2bcff12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:35.790748 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607b8ab-b969-4c8c-8950-5550f2bcff12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:35.792733 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f607b8ab-b969-4c8c-8950-5550f2bcff12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:35.823286 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-58a6079b-8dd8-4ed0-b306-62ea4a1de6fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c7a56725a124d4da713e3104d596acb', 'request_id': 'req-58a6079b-8dd8-4ed0-b306-62ea4a1de6fc'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:35.839990 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-58a6079b-8dd8-4ed0-b306-62ea4a1de6fc admin admin] The token cache is being invalidate because project 2c7a56725a124d4da713e3104d596acb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:06:35.840815 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-58a6079b-8dd8-4ed0-b306-62ea4a1de6fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:35.842911 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-58a6079b-8dd8-4ed0-b306-62ea4a1de6fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:35.844882 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1062/2111] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:06:35 2022] DELETE /identity/v3/projects/2c7a56725a124d4da713e3104d596acb => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:06:35.851001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e323029-0f24-44ab-a477-b1caff4410fb None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:35.851556 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e323029-0f24-44ab-a477-b1caff4410fb None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:35.852010 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e323029-0f24-44ab-a477-b1caff4410fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:35.877543 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-6e323029-0f24-44ab-a477-b1caff4410fb None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:35.878175 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f607b8ab-b969-4c8c-8950-5550f2bcff12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:35.878796 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6e323029-0f24-44ab-a477-b1caff4410fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:35.880180 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f607b8ab-b969-4c8c-8950-5550f2bcff12 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:35.919729 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1050/2112] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:06:35.972453 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1063/2113] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:35.980996 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4f8c2-bc6a-42ae-a838-408b88a29871 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:35.984866 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4f8c2-bc6a-42ae-a838-408b88a29871 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:35.984866 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4f8c2-bc6a-42ae-a838-408b88a29871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:36.007520 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-bdf4f8c2-bc6a-42ae-a838-408b88a29871 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:36.009010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-bdf4f8c2-bc6a-42ae-a838-408b88a29871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:36.019603 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1051/2114] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:36.024717 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e323029-0f24-44ab-a477-b1caff4410fb None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:36.037407 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e323029-0f24-44ab-a477-b1caff4410fb None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:36.043993 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ef97f-c691-49d0-8f1d-11f08f74e70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:36.045076 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ef97f-c691-49d0-8f1d-11f08f74e70e admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:36.045854 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ef97f-c691-49d0-8f1d-11f08f74e70e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:36.046312 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ef97f-c691-49d0-8f1d-11f08f74e70e admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:36.047488 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ef97f-c691-49d0-8f1d-11f08f74e70e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:06:36.047913 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ef97f-c691-49d0-8f1d-11f08f74e70e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:36.049948 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ef97f-c691-49d0-8f1d-11f08f74e70e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:36.067608 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1064/2115] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:06:36.072293 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4f8c2-bc6a-42ae-a838-408b88a29871 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:36.085070 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4f8c2-bc6a-42ae-a838-408b88a29871 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:36.092394 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66b822-2380-49de-9e22-dce14961fb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:36.093374 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66b822-2380-49de-9e22-dce14961fb8a admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:36.093845 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66b822-2380-49de-9e22-dce14961fb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:36.094442 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66b822-2380-49de-9e22-dce14961fb8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:36.107957 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a66b822-2380-49de-9e22-dce14961fb8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:36.110736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a66b822-2380-49de-9e22-dce14961fb8a admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:36.216352 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1052/2116] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:06:36 2022] POST /identity/v3/projects => generated 375 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:36.220443 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ef97f-c691-49d0-8f1d-11f08f74e70e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:36.233805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ef97f-c691-49d0-8f1d-11f08f74e70e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:36.318588 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f4cba-16a0-4379-8815-5cdc547d8546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:36.319522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f4cba-16a0-4379-8815-5cdc547d8546 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:36.319813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f4cba-16a0-4379-8815-5cdc547d8546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:36.320109 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f4cba-16a0-4379-8815-5cdc547d8546 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:36.320804 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f4cba-16a0-4379-8815-5cdc547d8546 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:36.325796 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f4cba-16a0-4379-8815-5cdc547d8546 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:36.441911 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1065/2117] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:06:36 2022] POST /identity/v3/users => generated 288 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:06:36.446461 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66b822-2380-49de-9e22-dce14961fb8a admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:36.459415 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66b822-2380-49de-9e22-dce14961fb8a admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:36.464594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605075e-7c10-428c-b8e6-982744e19109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:36.465829 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605075e-7c10-428c-b8e6-982744e19109 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:36.466272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605075e-7c10-428c-b8e6-982744e19109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:36.466839 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605075e-7c10-428c-b8e6-982744e19109 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:36.467815 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4605075e-7c10-428c-b8e6-982744e19109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:36.469833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4605075e-7c10-428c-b8e6-982744e19109 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:36.511595 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1053/2118] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:36.513079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f4cba-16a0-4379-8815-5cdc547d8546 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:36.520282 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:36.529431 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f4cba-16a0-4379-8815-5cdc547d8546 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:36.535524 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e4a40a-8f14-4991-a346-137d9dd8a184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:36.536306 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4a40a-8f14-4991-a346-137d9dd8a184 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:36.536788 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4a40a-8f14-4991-a346-137d9dd8a184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:36.537214 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4a40a-8f14-4991-a346-137d9dd8a184 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:36.538049 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e4a40a-8f14-4991-a346-137d9dd8a184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:36.539835 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e4a40a-8f14-4991-a346-137d9dd8a184 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:36.559036 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605075e-7c10-428c-b8e6-982744e19109 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:36.564209 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1066/2119] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:36.571859 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605075e-7c10-428c-b8e6-982744e19109 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:36.610563 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970aff4f-ad2b-4244-848b-bc32c6617500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:36.611238 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970aff4f-ad2b-4244-848b-bc32c6617500 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:36.611523 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970aff4f-ad2b-4244-848b-bc32c6617500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:36.611801 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970aff4f-ad2b-4244-848b-bc32c6617500 admin admin] PATH_INFO: `/v3/projects/96cc1c3449c84fdc817b2fded2250fb8/users/bfcd8bf465e741748ae383e00d036ae0/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:36.655978 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970aff4f-ad2b-4244-848b-bc32c6617500 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96cc1c3449c84fdc817b2fded2250fb8, user_id=bfcd8bf465e741748ae383e00d036ae0, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:36.660355 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970aff4f-ad2b-4244-848b-bc32c6617500 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:36.772782 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1054/2120] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:36 2022] PUT /identity/v3/projects/96cc1c3449c84fdc817b2fded2250fb8/users/bfcd8bf465e741748ae383e00d036ae0/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:06:36.783276 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08916dad-a970-4135-b500-af2493826c1e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:36.783597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08916dad-a970-4135-b500-af2493826c1e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:36.783828 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08916dad-a970-4135-b500-af2493826c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:36.817483 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-08916dad-a970-4135-b500-af2493826c1e None None] MFA Rules not processed for user `bfcd8bf465e741748ae383e00d036ae0`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:36.819406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-08916dad-a970-4135-b500-af2493826c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:36.915611 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1067/2121] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:36.931146 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970aff4f-ad2b-4244-848b-bc32c6617500 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:36.933617 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-970aff4f-ad2b-4244-848b-bc32c6617500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:37.048694 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970aff4f-ad2b-4244-848b-bc32c6617500 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:37.053102 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ca5d5-54ca-4d8f-8a3b-97dcbd193dc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:37.053710 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ca5d5-54ca-4d8f-8a3b-97dcbd193dc1 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:37.053991 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ca5d5-54ca-4d8f-8a3b-97dcbd193dc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:37.054260 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ca5d5-54ca-4d8f-8a3b-97dcbd193dc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:37.063367 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01ca5d5-54ca-4d8f-8a3b-97dcbd193dc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:37.064830 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01ca5d5-54ca-4d8f-8a3b-97dcbd193dc1 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:37.074791 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1055/2122] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:06:37.555357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:37.724509 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08916dad-a970-4135-b500-af2493826c1e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:37.726943 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-08916dad-a970-4135-b500-af2493826c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:37.857897 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08916dad-a970-4135-b500-af2493826c1e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:37.863911 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b87756-4e82-409c-84aa-987e891fbc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:37.864592 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b87756-4e82-409c-84aa-987e891fbc0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:37.867843 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b87756-4e82-409c-84aa-987e891fbc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:37.867843 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b87756-4e82-409c-84aa-987e891fbc0a admin admin] PATH_INFO: `/v3/users/5e71d02e626f42bda29c4f6afe82bb0b` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:37.867843 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b87756-4e82-409c-84aa-987e891fbc0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e71d02e626f42bda29c4f6afe82bb0b)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:37.869648 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b87756-4e82-409c-84aa-987e891fbc0a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:38.338251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-77b87756-4e82-409c-84aa-987e891fbc0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e71d02e626f42bda29c4f6afe82bb0b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:38.339120 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-77b87756-4e82-409c-84aa-987e891fbc0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e71d02e626f42bda29c4f6afe82bb0b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:38.339887 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-77b87756-4e82-409c-84aa-987e891fbc0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e71d02e626f42bda29c4f6afe82bb0b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:38.351803 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:38.356806 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-77b87756-4e82-409c-84aa-987e891fbc0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e71d02e626f42bda29c4f6afe82bb0b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:38.377528 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-77b87756-4e82-409c-84aa-987e891fbc0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e71d02e626f42bda29c4f6afe82bb0b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:38.444969 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1068/2123] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:06:37 2022] DELETE /identity/v3/users/5e71d02e626f42bda29c4f6afe82bb0b => generated 0 bytes in 721 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:38.449492 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760c85f-ee8d-4c5d-9977-c16965d0bc24 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:38.449843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760c85f-ee8d-4c5d-9977-c16965d0bc24 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:38.450193 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760c85f-ee8d-4c5d-9977-c16965d0bc24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:38.474762 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-0760c85f-ee8d-4c5d-9977-c16965d0bc24 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:38.491094 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0760c85f-ee8d-4c5d-9977-c16965d0bc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:38.611213 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1056/2124] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:06:38.622032 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b87756-4e82-409c-84aa-987e891fbc0a admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:38.623916 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-77b87756-4e82-409c-84aa-987e891fbc0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:38.730618 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b87756-4e82-409c-84aa-987e891fbc0a admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:38.737162 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6006b70b-49a1-43e8-ad64-26d120ef1b78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:38.738783 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006b70b-49a1-43e8-ad64-26d120ef1b78 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:38.739236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006b70b-49a1-43e8-ad64-26d120ef1b78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:38.739656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006b70b-49a1-43e8-ad64-26d120ef1b78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:38.750742 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6006b70b-49a1-43e8-ad64-26d120ef1b78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:38.752807 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6006b70b-49a1-43e8-ad64-26d120ef1b78 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:38.765211 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1069/2125] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:06:39.146747 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760c85f-ee8d-4c5d-9977-c16965d0bc24 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:39.151458 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0760c85f-ee8d-4c5d-9977-c16965d0bc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:39.219522 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760c85f-ee8d-4c5d-9977-c16965d0bc24 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:39.223996 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df44c4b-8fe5-4fe1-9935-7296fc4c4c18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:39.226904 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df44c4b-8fe5-4fe1-9935-7296fc4c4c18 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:39.226904 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df44c4b-8fe5-4fe1-9935-7296fc4c4c18 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:39.226904 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df44c4b-8fe5-4fe1-9935-7296fc4c4c18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:39.228954 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8df44c4b-8fe5-4fe1-9935-7296fc4c4c18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:39.322393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df44c4b-8fe5-4fe1-9935-7296fc4c4c18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:39.324357 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df44c4b-8fe5-4fe1-9935-7296fc4c4c18 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:39.340970 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1057/2126] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:39 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:06:39.552546 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6006b70b-49a1-43e8-ad64-26d120ef1b78 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:39.556889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6006b70b-49a1-43e8-ad64-26d120ef1b78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:39.582448 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6006b70b-49a1-43e8-ad64-26d120ef1b78 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:39.599624 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc31c48-0ee3-4c24-9b19-aab97f9889bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:39.600474 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc31c48-0ee3-4c24-9b19-aab97f9889bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:39.600661 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc31c48-0ee3-4c24-9b19-aab97f9889bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:39.600936 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc31c48-0ee3-4c24-9b19-aab97f9889bf admin admin] PATH_INFO: `/v3/projects/d31d8ff2123a4119837f06a5120d15b5` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:39.603538 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc31c48-0ee3-4c24-9b19-aab97f9889bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d31d8ff2123a4119837f06a5120d15b5)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:39.605752 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc31c48-0ee3-4c24-9b19-aab97f9889bf admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:39.766712 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4cc31c48-0ee3-4c24-9b19-aab97f9889bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd31d8ff2123a4119837f06a5120d15b5', 'request_id': 'req-4cc31c48-0ee3-4c24-9b19-aab97f9889bf'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:39.775651 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4cc31c48-0ee3-4c24-9b19-aab97f9889bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd31d8ff2123a4119837f06a5120d15b5', 'request_id': 'req-4cc31c48-0ee3-4c24-9b19-aab97f9889bf'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:39.835473 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4cc31c48-0ee3-4c24-9b19-aab97f9889bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd31d8ff2123a4119837f06a5120d15b5', 'request_id': 'req-4cc31c48-0ee3-4c24-9b19-aab97f9889bf'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:39.850820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4cc31c48-0ee3-4c24-9b19-aab97f9889bf admin admin] The token cache is being invalidate because project d31d8ff2123a4119837f06a5120d15b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:06:39.851120 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4cc31c48-0ee3-4c24-9b19-aab97f9889bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:39.851921 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4cc31c48-0ee3-4c24-9b19-aab97f9889bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:39.853684 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1070/2127] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:06:39 2022] DELETE /identity/v3/projects/d31d8ff2123a4119837f06a5120d15b5 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:39.857817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df44c4b-8fe5-4fe1-9935-7296fc4c4c18 service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:39.885279 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df44c4b-8fe5-4fe1-9935-7296fc4c4c18 service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:39.894013 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:39.976256 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c94bf-3b9f-4bba-8498-ff2378aba003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:39.977163 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c94bf-3b9f-4bba-8498-ff2378aba003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:39.978021 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c94bf-3b9f-4bba-8498-ff2378aba003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:39.978455 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c94bf-3b9f-4bba-8498-ff2378aba003 admin admin] PATH_INFO: `/v3/users/a0e0cd355ab247e9b94bc1209753f0a5` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:39.986627 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5c94bf-3b9f-4bba-8498-ff2378aba003 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0e0cd355ab247e9b94bc1209753f0a5)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:39.994553 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5c94bf-3b9f-4bba-8498-ff2378aba003 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:40.148981 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542fe84-d1e3-4612-8610-d1fa554f53eb None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:40.149500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542fe84-d1e3-4612-8610-d1fa554f53eb None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:40.149783 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542fe84-d1e3-4612-8610-d1fa554f53eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:40.181491 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d542fe84-d1e3-4612-8610-d1fa554f53eb None None] MFA Rules not processed for user `6bcc5313cdc04538a6b0f344cd369aab`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:40.182731 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d542fe84-d1e3-4612-8610-d1fa554f53eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:40.276012 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ae5c94bf-3b9f-4bba-8498-ff2378aba003 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0e0cd355ab247e9b94bc1209753f0a5'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:40.277579 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ae5c94bf-3b9f-4bba-8498-ff2378aba003 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0e0cd355ab247e9b94bc1209753f0a5'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:40.294378 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1071/2128] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:40 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:06:40.298656 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ae5c94bf-3b9f-4bba-8498-ff2378aba003 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0e0cd355ab247e9b94bc1209753f0a5'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:40.304940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542fe84-d1e3-4612-8610-d1fa554f53eb None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:40.306850 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d542fe84-d1e3-4612-8610-d1fa554f53eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:40.317704 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ae5c94bf-3b9f-4bba-8498-ff2378aba003 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0e0cd355ab247e9b94bc1209753f0a5'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:40.318477 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ae5c94bf-3b9f-4bba-8498-ff2378aba003 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0e0cd355ab247e9b94bc1209753f0a5'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:40.409525 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542fe84-d1e3-4612-8610-d1fa554f53eb None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:40.416325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eff8a3-3b61-4ae4-b8a0-167644d1e3aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:40.416946 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eff8a3-3b61-4ae4-b8a0-167644d1e3aa service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:40.417226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eff8a3-3b61-4ae4-b8a0-167644d1e3aa service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:40.417501 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eff8a3-3b61-4ae4-b8a0-167644d1e3aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:40.420521 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e7eff8a3-3b61-4ae4-b8a0-167644d1e3aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:40.435902 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1058/2129] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:06:39 2022] DELETE /identity/v3/users/a0e0cd355ab247e9b94bc1209753f0a5 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:06:40.525271 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eff8a3-3b61-4ae4-b8a0-167644d1e3aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:40.527457 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eff8a3-3b61-4ae4-b8a0-167644d1e3aa service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:40.539262 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:40.544506 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1072/2130] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:06:41.065764 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c94bf-3b9f-4bba-8498-ff2378aba003 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:41.067239 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ae5c94bf-3b9f-4bba-8498-ff2378aba003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:41.150696 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c94bf-3b9f-4bba-8498-ff2378aba003 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:41.155807 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb54cf-542b-47f5-965a-69584fe75447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:41.156481 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb54cf-542b-47f5-965a-69584fe75447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:41.156761 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb54cf-542b-47f5-965a-69584fe75447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:41.157165 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb54cf-542b-47f5-965a-69584fe75447 admin admin] PATH_INFO: `/v3/projects/f766d1760282466d8ef637b17c32fc0b` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:41.158332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb54cf-542b-47f5-965a-69584fe75447 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f766d1760282466d8ef637b17c32fc0b)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:41.160528 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb54cf-542b-47f5-965a-69584fe75447 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:41.295751 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-bdeb54cf-542b-47f5-965a-69584fe75447 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f766d1760282466d8ef637b17c32fc0b', 'request_id': 'req-bdeb54cf-542b-47f5-965a-69584fe75447'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:41.359562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:41.361835 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-bdeb54cf-542b-47f5-965a-69584fe75447 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f766d1760282466d8ef637b17c32fc0b', 'request_id': 'req-bdeb54cf-542b-47f5-965a-69584fe75447'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:41.370597 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-bdeb54cf-542b-47f5-965a-69584fe75447 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f766d1760282466d8ef637b17c32fc0b', 'request_id': 'req-bdeb54cf-542b-47f5-965a-69584fe75447'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:41.381790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-bdeb54cf-542b-47f5-965a-69584fe75447 admin admin] The token cache is being invalidate because project f766d1760282466d8ef637b17c32fc0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:06:41.381790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-bdeb54cf-542b-47f5-965a-69584fe75447 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:41.382523 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-bdeb54cf-542b-47f5-965a-69584fe75447 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:41.388410 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1059/2131] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:06:41 2022] DELETE /identity/v3/projects/f766d1760282466d8ef637b17c32fc0b => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:41.390078 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd2ab7-921c-4038-a829-bc251b0800e6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:41.390366 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd2ab7-921c-4038-a829-bc251b0800e6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:41.390642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd2ab7-921c-4038-a829-bc251b0800e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:41.411269 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-89bd2ab7-921c-4038-a829-bc251b0800e6 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:41.412590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-89bd2ab7-921c-4038-a829-bc251b0800e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:41.481029 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1073/2132] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:41.489574 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420cb6a-d79b-464d-b37f-c5d447bb5855 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:41.490425 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420cb6a-d79b-464d-b37f-c5d447bb5855 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:41.492694 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420cb6a-d79b-464d-b37f-c5d447bb5855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:41.518058 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-c420cb6a-d79b-464d-b37f-c5d447bb5855 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:41.519763 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c420cb6a-d79b-464d-b37f-c5d447bb5855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:41.531272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1060/2133] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:41.535711 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bd2ab7-921c-4038-a829-bc251b0800e6 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:41.547634 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bd2ab7-921c-4038-a829-bc251b0800e6 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:41.554315 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044d2e1-4f5a-4a4b-9959-f36039cbd855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:41.555127 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044d2e1-4f5a-4a4b-9959-f36039cbd855 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:41.562451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044d2e1-4f5a-4a4b-9959-f36039cbd855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:41.562451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044d2e1-4f5a-4a4b-9959-f36039cbd855 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:41.562451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2044d2e1-4f5a-4a4b-9959-f36039cbd855 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:06:41.562451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2044d2e1-4f5a-4a4b-9959-f36039cbd855 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:41.564444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2044d2e1-4f5a-4a4b-9959-f36039cbd855 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:41.578659 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1074/2134] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:06:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:06:41.583190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420cb6a-d79b-464d-b37f-c5d447bb5855 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:41.593477 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420cb6a-d79b-464d-b37f-c5d447bb5855 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:41.600457 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f4639-2fb5-48cd-a148-b993a8cc345a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:41.601269 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f4639-2fb5-48cd-a148-b993a8cc345a admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:41.601549 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f4639-2fb5-48cd-a148-b993a8cc345a admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:41.601877 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f4639-2fb5-48cd-a148-b993a8cc345a admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:41.602545 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f4639-2fb5-48cd-a148-b993a8cc345a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:41.604593 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f4639-2fb5-48cd-a148-b993a8cc345a admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:41.716771 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1061/2135] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:06:41 2022] POST /identity/v3/projects => generated 377 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:41.721078 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044d2e1-4f5a-4a4b-9959-f36039cbd855 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:41.732933 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044d2e1-4f5a-4a4b-9959-f36039cbd855 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:41.831356 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171ac24-fb69-4993-99cd-0876b51c67cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:41.831974 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171ac24-fb69-4993-99cd-0876b51c67cd admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:41.832318 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171ac24-fb69-4993-99cd-0876b51c67cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:41.832625 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171ac24-fb69-4993-99cd-0876b51c67cd admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:41.833260 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c171ac24-fb69-4993-99cd-0876b51c67cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:41.838149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c171ac24-fb69-4993-99cd-0876b51c67cd admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:41.948325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1075/2136] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:06:41 2022] POST /identity/v3/users => generated 289 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:41.952832 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f4639-2fb5-48cd-a148-b993a8cc345a admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:41.967846 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f4639-2fb5-48cd-a148-b993a8cc345a admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:41.976512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709493fb-48ee-4327-93da-031b6a4f4420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:41.977127 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709493fb-48ee-4327-93da-031b6a4f4420 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:41.977799 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709493fb-48ee-4327-93da-031b6a4f4420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:41.978078 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709493fb-48ee-4327-93da-031b6a4f4420 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:41.978744 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709493fb-48ee-4327-93da-031b6a4f4420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:41.980603 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709493fb-48ee-4327-93da-031b6a4f4420 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:41.998974 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171ac24-fb69-4993-99cd-0876b51c67cd admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:41.999735 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1062/2137] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:42.011520 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171ac24-fb69-4993-99cd-0876b51c67cd admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:42.019356 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d93acd-a44d-4ebb-9664-b975fd2ddc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:42.020099 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d93acd-a44d-4ebb-9664-b975fd2ddc89 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:42.020525 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d93acd-a44d-4ebb-9664-b975fd2ddc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:42.024663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d93acd-a44d-4ebb-9664-b975fd2ddc89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:42.024663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d93acd-a44d-4ebb-9664-b975fd2ddc89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:42.024663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d93acd-a44d-4ebb-9664-b975fd2ddc89 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:42.037191 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1076/2138] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:06:42.041419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709493fb-48ee-4327-93da-031b6a4f4420 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:42.057720 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709493fb-48ee-4327-93da-031b6a4f4420 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:42.063904 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e296a6-cd06-40d7-9799-f8ed2cebe57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:42.066057 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e296a6-cd06-40d7-9799-f8ed2cebe57f admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:42.066057 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e296a6-cd06-40d7-9799-f8ed2cebe57f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:42.066057 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e296a6-cd06-40d7-9799-f8ed2cebe57f admin admin] PATH_INFO: `/v3/projects/ce69fa81b3ff401ab7cbeafdb0bdfa11/users/f22db2ce76984ba3a7013439fa053018/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:42.093990 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e296a6-cd06-40d7-9799-f8ed2cebe57f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce69fa81b3ff401ab7cbeafdb0bdfa11, user_id=f22db2ce76984ba3a7013439fa053018, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:42.093990 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e296a6-cd06-40d7-9799-f8ed2cebe57f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:42.202704 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1063/2139] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:42 2022] PUT /identity/v3/projects/ce69fa81b3ff401ab7cbeafdb0bdfa11/users/f22db2ce76984ba3a7013439fa053018/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:42.215091 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8866a-4b9f-4242-97eb-94920a4c81b3 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:42.215383 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8866a-4b9f-4242-97eb-94920a4c81b3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:42.215807 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8866a-4b9f-4242-97eb-94920a4c81b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:42.251904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b2a8866a-4b9f-4242-97eb-94920a4c81b3 None None] MFA Rules not processed for user `f22db2ce76984ba3a7013439fa053018`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:42.253593 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b2a8866a-4b9f-4242-97eb-94920a4c81b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:42.340121 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1077/2140] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:42.344163 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e296a6-cd06-40d7-9799-f8ed2cebe57f admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:42.461439 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e296a6-cd06-40d7-9799-f8ed2cebe57f admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:42.466635 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6a3e8c-96ee-42ee-9f68-92921247a1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:42.467346 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a3e8c-96ee-42ee-9f68-92921247a1fb admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:42.467685 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a3e8c-96ee-42ee-9f68-92921247a1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:42.468045 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a3e8c-96ee-42ee-9f68-92921247a1fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:42.468782 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6a3e8c-96ee-42ee-9f68-92921247a1fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:42.470816 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6a3e8c-96ee-42ee-9f68-92921247a1fb admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:42.595303 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1064/2141] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:06:42 2022] POST /identity/v3/projects => generated 377 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:42.596505 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8866a-4b9f-4242-97eb-94920a4c81b3 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:42.608314 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8866a-4b9f-4242-97eb-94920a4c81b3 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:42.670068 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:42.694355 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077996f-6239-4a00-b957-58cd83d9df27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:42.695167 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077996f-6239-4a00-b957-58cd83d9df27 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:42.695564 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077996f-6239-4a00-b957-58cd83d9df27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:42.695948 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077996f-6239-4a00-b957-58cd83d9df27 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:42.701974 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077996f-6239-4a00-b957-58cd83d9df27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:42.701974 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077996f-6239-4a00-b957-58cd83d9df27 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:42.743977 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:42.777943 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1078/2142] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:06:42 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:42.781966 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6a3e8c-96ee-42ee-9f68-92921247a1fb admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:42.793400 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6a3e8c-96ee-42ee-9f68-92921247a1fb admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:42.799523 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df53d130-6bc8-4df9-91d7-ebbfbb22bce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:42.800471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53d130-6bc8-4df9-91d7-ebbfbb22bce4 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:42.800910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53d130-6bc8-4df9-91d7-ebbfbb22bce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:42.801187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53d130-6bc8-4df9-91d7-ebbfbb22bce4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:42.801842 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df53d130-6bc8-4df9-91d7-ebbfbb22bce4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:42.804184 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df53d130-6bc8-4df9-91d7-ebbfbb22bce4 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:42.818731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1065/2143] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:06:42.822993 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077996f-6239-4a00-b957-58cd83d9df27 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:42.833977 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077996f-6239-4a00-b957-58cd83d9df27 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:42.836665 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df53d130-6bc8-4df9-91d7-ebbfbb22bce4 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:42.838360 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-df53d130-6bc8-4df9-91d7-ebbfbb22bce4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:42.838964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a3aa0-52ca-4183-9eb4-dd84b5808424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:42.839565 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a3aa0-52ca-4183-9eb4-dd84b5808424 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:42.840465 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a3aa0-52ca-4183-9eb4-dd84b5808424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:42.840713 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a3aa0-52ca-4183-9eb4-dd84b5808424 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:42.841359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a3aa0-52ca-4183-9eb4-dd84b5808424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:42.842941 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a3aa0-52ca-4183-9eb4-dd84b5808424 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:42.851832 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df53d130-6bc8-4df9-91d7-ebbfbb22bce4 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:42.853672 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1079/2144] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:42.857324 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a3aa0-52ca-4183-9eb4-dd84b5808424 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:42.859701 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979297f5-9de8-4c0b-b1b6-779af08c7bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:42.860409 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979297f5-9de8-4c0b-b1b6-779af08c7bb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:42.860665 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979297f5-9de8-4c0b-b1b6-779af08c7bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:42.860939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979297f5-9de8-4c0b-b1b6-779af08c7bb8 admin admin] PATH_INFO: `/v3/users/bfcd8bf465e741748ae383e00d036ae0` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:42.862138 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979297f5-9de8-4c0b-b1b6-779af08c7bb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfcd8bf465e741748ae383e00d036ae0)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:42.864104 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979297f5-9de8-4c0b-b1b6-779af08c7bb8 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:42.867912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a3aa0-52ca-4183-9eb4-dd84b5808424 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:42.873172 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098fc3d-5bc0-42af-b76a-d87d576a248f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:42.875015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098fc3d-5bc0-42af-b76a-d87d576a248f admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:42.875015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098fc3d-5bc0-42af-b76a-d87d576a248f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:42.875015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098fc3d-5bc0-42af-b76a-d87d576a248f admin admin] PATH_INFO: `/v3/projects/828937e1a4fa4d0094ab782fea9c3e36/users/ca00ba6208e941638da71ec4dba57e71/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:42.890274 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c098fc3d-5bc0-42af-b76a-d87d576a248f admin admin] RBAC: Authorizing `identity:create_grant(project_id=828937e1a4fa4d0094ab782fea9c3e36, user_id=ca00ba6208e941638da71ec4dba57e71, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:42.892962 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c098fc3d-5bc0-42af-b76a-d87d576a248f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:42.993790 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1080/2145] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:42 2022] PUT /identity/v3/projects/828937e1a4fa4d0094ab782fea9c3e36/users/ca00ba6208e941638da71ec4dba57e71/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:06:43.004490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf36c05-c7b5-408f-a5e4-66a1d3bc2072 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:43.004853 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf36c05-c7b5-408f-a5e4-66a1d3bc2072 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:43.005329 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf36c05-c7b5-408f-a5e4-66a1d3bc2072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:43.040567 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ddf36c05-c7b5-408f-a5e4-66a1d3bc2072 None None] MFA Rules not processed for user `ca00ba6208e941638da71ec4dba57e71`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:43.041774 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ddf36c05-c7b5-408f-a5e4-66a1d3bc2072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:43.065495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-979297f5-9de8-4c0b-b1b6-779af08c7bb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfcd8bf465e741748ae383e00d036ae0'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:43.066216 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-979297f5-9de8-4c0b-b1b6-779af08c7bb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfcd8bf465e741748ae383e00d036ae0'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:43.075706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-979297f5-9de8-4c0b-b1b6-779af08c7bb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfcd8bf465e741748ae383e00d036ae0'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:43.094666 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-979297f5-9de8-4c0b-b1b6-779af08c7bb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfcd8bf465e741748ae383e00d036ae0'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:43.095679 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-979297f5-9de8-4c0b-b1b6-779af08c7bb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfcd8bf465e741748ae383e00d036ae0'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:43.137794 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1081/2146] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:06:43.142764 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf36c05-c7b5-408f-a5e4-66a1d3bc2072 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:43.144407 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ddf36c05-c7b5-408f-a5e4-66a1d3bc2072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:43.189374 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1066/2147] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:06:42 2022] DELETE /identity/v3/users/bfcd8bf465e741748ae383e00d036ae0 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:06:43.205099 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c7962-d9e4-4807-8a11-699f3be2af53 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:43.205390 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c7962-d9e4-4807-8a11-699f3be2af53 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:43.205664 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c7962-d9e4-4807-8a11-699f3be2af53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:43.237316 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf36c05-c7b5-408f-a5e4-66a1d3bc2072 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:43.252155 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-9a6c7962-d9e4-4807-8a11-699f3be2af53 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:43.256967 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-9a6c7962-d9e4-4807-8a11-699f3be2af53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:43.335331 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09deb029-0f09-43d3-8b80-34fbb91afbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:43.336978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09deb029-0f09-43d3-8b80-34fbb91afbda admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:43.338140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09deb029-0f09-43d3-8b80-34fbb91afbda admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:43.338140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09deb029-0f09-43d3-8b80-34fbb91afbda admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:43.338322 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09deb029-0f09-43d3-8b80-34fbb91afbda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:43.340893 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09deb029-0f09-43d3-8b80-34fbb91afbda admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:43.371490 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1067/2148] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:43.381575 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c7962-d9e4-4807-8a11-699f3be2af53 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:43.382909 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-9a6c7962-d9e4-4807-8a11-699f3be2af53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:43.435750 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1082/2149] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:06:43 2022] POST /identity/v3/projects => generated 377 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:43.437796 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09deb029-0f09-43d3-8b80-34fbb91afbda admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:43.470084 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c7962-d9e4-4807-8a11-699f3be2af53 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:43.524388 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09deb029-0f09-43d3-8b80-34fbb91afbda admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:43.529680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d1411-15fc-4b14-ba3a-29cb3e1b72cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:43.530300 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d1411-15fc-4b14-ba3a-29cb3e1b72cd admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:43.530613 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d1411-15fc-4b14-ba3a-29cb3e1b72cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:43.530893 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d1411-15fc-4b14-ba3a-29cb3e1b72cd admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:43.531533 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561d1411-15fc-4b14-ba3a-29cb3e1b72cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:43.533557 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561d1411-15fc-4b14-ba3a-29cb3e1b72cd admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:43.534475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29602237-3a4f-4894-9100-6588755468c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:43.535282 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29602237-3a4f-4894-9100-6588755468c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:43.535568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29602237-3a4f-4894-9100-6588755468c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:43.535843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29602237-3a4f-4894-9100-6588755468c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:43.545921 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29602237-3a4f-4894-9100-6588755468c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:43.547366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29602237-3a4f-4894-9100-6588755468c2 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:43.557142 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1068/2150] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:06:43.647101 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1083/2151] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:06:43 2022] POST /identity/v3/users => generated 295 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:06:43.651255 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29602237-3a4f-4894-9100-6588755468c2 service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:43.660947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29602237-3a4f-4894-9100-6588755468c2 service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:43.666771 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01658d9d-ad58-4695-a97e-ecd76a9a7534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:43.667387 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01658d9d-ad58-4695-a97e-ecd76a9a7534 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:43.667813 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01658d9d-ad58-4695-a97e-ecd76a9a7534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:43.668254 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01658d9d-ad58-4695-a97e-ecd76a9a7534 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:43.668911 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01658d9d-ad58-4695-a97e-ecd76a9a7534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:43.670572 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01658d9d-ad58-4695-a97e-ecd76a9a7534 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:43.684255 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1069/2152] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:43.685377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d1411-15fc-4b14-ba3a-29cb3e1b72cd admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:43.688225 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209fd7c-5f40-4afc-93b9-ac15c9c279dc None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:43.688581 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209fd7c-5f40-4afc-93b9-ac15c9c279dc None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:43.688932 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209fd7c-5f40-4afc-93b9-ac15c9c279dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:43.696277 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d1411-15fc-4b14-ba3a-29cb3e1b72cd admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:43.701078 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86099daa-0bc6-4db8-9a2f-39f13e4152e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:43.701997 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86099daa-0bc6-4db8-9a2f-39f13e4152e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:43.709301 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86099daa-0bc6-4db8-9a2f-39f13e4152e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:43.709301 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86099daa-0bc6-4db8-9a2f-39f13e4152e8 admin admin] PATH_INFO: `/v3/projects/170bd8b796af47fbacee0b05b65dcab0/users/a53f72887f114989b8c013b87ac0be04/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:43.710816 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-3209fd7c-5f40-4afc-93b9-ac15c9c279dc None None] MFA Rules not processed for user `b17adcb2be25427485d5004c7821e094`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:43.712505 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3209fd7c-5f40-4afc-93b9-ac15c9c279dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:43.721988 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86099daa-0bc6-4db8-9a2f-39f13e4152e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=170bd8b796af47fbacee0b05b65dcab0, user_id=a53f72887f114989b8c013b87ac0be04, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:43.724881 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86099daa-0bc6-4db8-9a2f-39f13e4152e8 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:43.801358 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1070/2153] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:06:43.830553 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1084/2154] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:43 2022] PUT /identity/v3/projects/170bd8b796af47fbacee0b05b65dcab0/users/a53f72887f114989b8c013b87ac0be04/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:06:43.835556 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3209fd7c-5f40-4afc-93b9-ac15c9c279dc None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:43.855296 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3209fd7c-5f40-4afc-93b9-ac15c9c279dc None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:43.953629 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2b03f-2a26-4c52-8720-976471840d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:43.954269 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2b03f-2a26-4c52-8720-976471840d9d admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:43.954554 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2b03f-2a26-4c52-8720-976471840d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:43.954828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2b03f-2a26-4c52-8720-976471840d9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:43.955500 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2b03f-2a26-4c52-8720-976471840d9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:43.957470 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2b03f-2a26-4c52-8720-976471840d9d admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:43.972463 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1071/2155] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:43.977038 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86099daa-0bc6-4db8-9a2f-39f13e4152e8 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:43.990503 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86099daa-0bc6-4db8-9a2f-39f13e4152e8 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:44.003619 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267b8c9-3789-4282-b3d1-336acf9e5a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:44.004844 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267b8c9-3789-4282-b3d1-336acf9e5a1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:44.005137 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267b8c9-3789-4282-b3d1-336acf9e5a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:44.007195 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267b8c9-3789-4282-b3d1-336acf9e5a1c admin admin] PATH_INFO: `/v3/projects/170bd8b796af47fbacee0b05b65dcab0/users/a53f72887f114989b8c013b87ac0be04/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:44.034955 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b267b8c9-3789-4282-b3d1-336acf9e5a1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=170bd8b796af47fbacee0b05b65dcab0, user_id=a53f72887f114989b8c013b87ac0be04, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:44.038623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b267b8c9-3789-4282-b3d1-336acf9e5a1c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:44.072191 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1085/2156] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:43 2022] PUT /identity/v3/projects/170bd8b796af47fbacee0b05b65dcab0/users/a53f72887f114989b8c013b87ac0be04/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:44.075789 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2e2d0-6fd7-46fb-837f-1f8205f7677e None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:44.085021 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2e2d0-6fd7-46fb-837f-1f8205f7677e None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:44.085021 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2e2d0-6fd7-46fb-837f-1f8205f7677e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:44.133686 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-f1d2e2d0-6fd7-46fb-837f-1f8205f7677e None None] MFA Rules not processed for user `a53f72887f114989b8c013b87ac0be04`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:44.135531 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f1d2e2d0-6fd7-46fb-837f-1f8205f7677e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:44.142650 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267b8c9-3789-4282-b3d1-336acf9e5a1c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:44.144662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b267b8c9-3789-4282-b3d1-336acf9e5a1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:44.260002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267b8c9-3789-4282-b3d1-336acf9e5a1c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:44.265511 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59922b8-f9ba-4e59-b3d5-f92fb73e5c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:44.266229 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59922b8-f9ba-4e59-b3d5-f92fb73e5c3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:44.266581 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59922b8-f9ba-4e59-b3d5-f92fb73e5c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:44.266865 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59922b8-f9ba-4e59-b3d5-f92fb73e5c3a admin admin] PATH_INFO: `/v3/projects/96cc1c3449c84fdc817b2fded2250fb8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:44.268046 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59922b8-f9ba-4e59-b3d5-f92fb73e5c3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=96cc1c3449c84fdc817b2fded2250fb8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:44.270395 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59922b8-f9ba-4e59-b3d5-f92fb73e5c3a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:44.298602 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1072/2157] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:44 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:44.431767 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d2e2d0-6fd7-46fb-837f-1f8205f7677e None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:44.439700 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f1d2e2d0-6fd7-46fb-837f-1f8205f7677e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:44.459890 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a59922b8-f9ba-4e59-b3d5-f92fb73e5c3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96cc1c3449c84fdc817b2fded2250fb8', 'request_id': 'req-a59922b8-f9ba-4e59-b3d5-f92fb73e5c3a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:44.477012 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a59922b8-f9ba-4e59-b3d5-f92fb73e5c3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96cc1c3449c84fdc817b2fded2250fb8', 'request_id': 'req-a59922b8-f9ba-4e59-b3d5-f92fb73e5c3a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:44.560021 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a59922b8-f9ba-4e59-b3d5-f92fb73e5c3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96cc1c3449c84fdc817b2fded2250fb8', 'request_id': 'req-a59922b8-f9ba-4e59-b3d5-f92fb73e5c3a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:44.568416 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d2e2d0-6fd7-46fb-837f-1f8205f7677e None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:44.577809 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a59922b8-f9ba-4e59-b3d5-f92fb73e5c3a admin admin] The token cache is being invalidate because project 96cc1c3449c84fdc817b2fded2250fb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:06:44.578105 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a59922b8-f9ba-4e59-b3d5-f92fb73e5c3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:44.580577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a59922b8-f9ba-4e59-b3d5-f92fb73e5c3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:44.585397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1086/2158] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:06:44 2022] DELETE /identity/v3/projects/96cc1c3449c84fdc817b2fded2250fb8 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:44.591880 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09edaf-b13b-4db3-aa24-2fb0a7dfc626 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:44.592216 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09edaf-b13b-4db3-aa24-2fb0a7dfc626 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:44.592538 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09edaf-b13b-4db3-aa24-2fb0a7dfc626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:44.636569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-7b09edaf-b13b-4db3-aa24-2fb0a7dfc626 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:44.638018 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7b09edaf-b13b-4db3-aa24-2fb0a7dfc626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:44.693380 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61293ea2-03f5-469b-bad8-62048768528e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:44.695131 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61293ea2-03f5-469b-bad8-62048768528e service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:44.695582 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61293ea2-03f5-469b-bad8-62048768528e service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:44.696006 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61293ea2-03f5-469b-bad8-62048768528e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:44.701337 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-61293ea2-03f5-469b-bad8-62048768528e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:44.718423 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1087/2159] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:44.727805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4f9e8-4938-4ec7-b8e6-ffa5a06bc8b6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:44.728159 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4f9e8-4938-4ec7-b8e6-ffa5a06bc8b6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:44.728542 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4f9e8-4938-4ec7-b8e6-ffa5a06bc8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:44.764273 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-12c4f9e8-4938-4ec7-b8e6-ffa5a06bc8b6 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:44.769255 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-12c4f9e8-4938-4ec7-b8e6-ffa5a06bc8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:44.774648 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1088/2160] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:44.779042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4f9e8-4938-4ec7-b8e6-ffa5a06bc8b6 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:44.791741 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4f9e8-4938-4ec7-b8e6-ffa5a06bc8b6 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:44.799819 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e373ce-632a-4589-b60b-3ea00ea93e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:44.799819 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e373ce-632a-4589-b60b-3ea00ea93e65 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:44.799819 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e373ce-632a-4589-b60b-3ea00ea93e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:44.799819 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e373ce-632a-4589-b60b-3ea00ea93e65 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:44.800148 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e373ce-632a-4589-b60b-3ea00ea93e65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:06:44.800148 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e373ce-632a-4589-b60b-3ea00ea93e65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:44.801635 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e373ce-632a-4589-b60b-3ea00ea93e65 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:44.811326 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61293ea2-03f5-469b-bad8-62048768528e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:44.812996 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61293ea2-03f5-469b-bad8-62048768528e service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:44.826506 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1089/2161] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:06:44.827753 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e373ce-632a-4589-b60b-3ea00ea93e65 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:44.845143 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e373ce-632a-4589-b60b-3ea00ea93e65 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:44.845961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1073/2162] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:06:44.855548 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0595cb-28ab-4ce1-8811-b021bb3b9875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:44.861031 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0595cb-28ab-4ce1-8811-b021bb3b9875 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:44.861356 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0595cb-28ab-4ce1-8811-b021bb3b9875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:44.861636 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0595cb-28ab-4ce1-8811-b021bb3b9875 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:44.862935 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0595cb-28ab-4ce1-8811-b021bb3b9875 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:44.865681 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0595cb-28ab-4ce1-8811-b021bb3b9875 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:44.893348 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:44.966449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:44.986258 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1090/2163] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:06:44 2022] POST /identity/v3/projects => generated 365 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:44.990010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61293ea2-03f5-469b-bad8-62048768528e service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:45.006247 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61293ea2-03f5-469b-bad8-62048768528e service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:45.132522 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ef14d6-5657-4c99-a1e3-e18f8420e556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:45.133814 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef14d6-5657-4c99-a1e3-e18f8420e556 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:45.138053 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef14d6-5657-4c99-a1e3-e18f8420e556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:45.141421 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef14d6-5657-4c99-a1e3-e18f8420e556 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:45.144298 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ef14d6-5657-4c99-a1e3-e18f8420e556 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:45.151740 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ef14d6-5657-4c99-a1e3-e18f8420e556 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:45.295976 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1074/2164] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:06:44 2022] POST /identity/v3/users => generated 283 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:45.299843 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0595cb-28ab-4ce1-8811-b021bb3b9875 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:45.314601 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0595cb-28ab-4ce1-8811-b021bb3b9875 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:45.320047 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446f3f1-dcbb-4001-aefd-f06ab4c792b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:45.320687 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446f3f1-dcbb-4001-aefd-f06ab4c792b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:45.320969 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446f3f1-dcbb-4001-aefd-f06ab4c792b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:45.321275 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446f3f1-dcbb-4001-aefd-f06ab4c792b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:45.322013 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8446f3f1-dcbb-4001-aefd-f06ab4c792b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:45.323575 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8446f3f1-dcbb-4001-aefd-f06ab4c792b6 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:45.344323 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1091/2165] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:06:45.349927 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ef14d6-5657-4c99-a1e3-e18f8420e556 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:45.368968 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ef14d6-5657-4c99-a1e3-e18f8420e556 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:45.379781 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d4c1c8-93bf-4af4-8e74-ab0f4655d73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:45.380981 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4c1c8-93bf-4af4-8e74-ab0f4655d73c admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:45.381263 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4c1c8-93bf-4af4-8e74-ab0f4655d73c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:45.383083 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4c1c8-93bf-4af4-8e74-ab0f4655d73c admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:45.383083 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d4c1c8-93bf-4af4-8e74-ab0f4655d73c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:45.384841 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d4c1c8-93bf-4af4-8e74-ab0f4655d73c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:45.398300 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1075/2166] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:45.401952 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446f3f1-dcbb-4001-aefd-f06ab4c792b6 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:45.412490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446f3f1-dcbb-4001-aefd-f06ab4c792b6 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:45.417470 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e9bf0-5939-45a1-93f1-9b159efedb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:45.418274 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e9bf0-5939-45a1-93f1-9b159efedb84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:45.418387 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e9bf0-5939-45a1-93f1-9b159efedb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:45.418618 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e9bf0-5939-45a1-93f1-9b159efedb84 admin admin] PATH_INFO: `/v3/projects/da98d88709a243ccabdb79a4524b68a7/users/3a9591d38f5146849ae15fdd5685c445/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:45.438242 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e9bf0-5939-45a1-93f1-9b159efedb84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da98d88709a243ccabdb79a4524b68a7, user_id=3a9591d38f5146849ae15fdd5685c445, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:45.441150 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e9bf0-5939-45a1-93f1-9b159efedb84 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:45.554189 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1092/2167] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:45 2022] PUT /identity/v3/projects/da98d88709a243ccabdb79a4524b68a7/users/3a9591d38f5146849ae15fdd5685c445/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:45.564555 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346a844-7c35-49de-9b4a-fb8be432bc11 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:45.564555 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346a844-7c35-49de-9b4a-fb8be432bc11 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:45.566160 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346a844-7c35-49de-9b4a-fb8be432bc11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:45.598266 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-6346a844-7c35-49de-9b4a-fb8be432bc11 None None] MFA Rules not processed for user `3a9591d38f5146849ae15fdd5685c445`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:45.599709 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-6346a844-7c35-49de-9b4a-fb8be432bc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:45.720924 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1076/2168] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:45.730792 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e9bf0-5939-45a1-93f1-9b159efedb84 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:45.733667 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c62e9bf0-5939-45a1-93f1-9b159efedb84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:45.844506 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e9bf0-5939-45a1-93f1-9b159efedb84 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:45.849737 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95b102-d2c9-48d2-b627-dfd211a5d562 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:45.850354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95b102-d2c9-48d2-b627-dfd211a5d562 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:45.850630 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95b102-d2c9-48d2-b627-dfd211a5d562 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:45.850898 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95b102-d2c9-48d2-b627-dfd211a5d562 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:45.865115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95b102-d2c9-48d2-b627-dfd211a5d562 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:45.866566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95b102-d2c9-48d2-b627-dfd211a5d562 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:45.894396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1093/2169] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:06:46.428636 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:46.795868 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:48.031997 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346a844-7c35-49de-9b4a-fb8be432bc11 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:48.033481 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-6346a844-7c35-49de-9b4a-fb8be432bc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:48.050576 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346a844-7c35-49de-9b4a-fb8be432bc11 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:48.057189 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bfb3d5-04e8-46f3-b61e-c301b9989a1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:48.057189 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfb3d5-04e8-46f3-b61e-c301b9989a1b service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:48.058132 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfb3d5-04e8-46f3-b61e-c301b9989a1b service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:48.058132 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfb3d5-04e8-46f3-b61e-c301b9989a1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:48.059750 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b7bfb3d5-04e8-46f3-b61e-c301b9989a1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:48.153365 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bfb3d5-04e8-46f3-b61e-c301b9989a1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:48.155456 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bfb3d5-04e8-46f3-b61e-c301b9989a1b service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:48.179783 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:48.180361 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1077/2170] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:06:49.292925 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:49.496709 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbf8e0-fe3e-4c2c-ae68-84110cd17ac3 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:49.497258 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbf8e0-fe3e-4c2c-ae68-84110cd17ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:49.497258 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbf8e0-fe3e-4c2c-ae68-84110cd17ac3 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:49.498577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1094/2171] 10.222.0.48 () {54 vars in 921 bytes} [Fri Sep 9 21:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:06:49.506945 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78467695-ff7b-4d1c-8467-984bfabd639f None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:49.507233 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78467695-ff7b-4d1c-8467-984bfabd639f None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:49.507508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78467695-ff7b-4d1c-8467-984bfabd639f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:49.527782 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-78467695-ff7b-4d1c-8467-984bfabd639f None None] MFA Rules not processed for user `2a4f6de2065d4fbea3fa141da760f708`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:49.529034 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-78467695-ff7b-4d1c-8467-984bfabd639f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:49.600697 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1078/2172] 10.222.0.48 () {58 vars in 1018 bytes} [Fri Sep 9 21:06:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:49.612208 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a7301-79de-4460-9f3a-c53a4ea8b076 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:49.612534 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a7301-79de-4460-9f3a-c53a4ea8b076 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:49.615046 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a7301-79de-4460-9f3a-c53a4ea8b076 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:49.615046 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1095/2173] 10.222.0.48 () {54 vars in 961 bytes} [Fri Sep 9 21:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:06:49.619662 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb292c-350b-4ae2-8165-90d2cefaac0f None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:49.619957 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb292c-350b-4ae2-8165-90d2cefaac0f None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:49.620371 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb292c-350b-4ae2-8165-90d2cefaac0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:49.641810 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-50bb292c-350b-4ae2-8165-90d2cefaac0f None None] MFA Rules not processed for user `2a4f6de2065d4fbea3fa141da760f708`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:49.643100 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-50bb292c-350b-4ae2-8165-90d2cefaac0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:49.655039 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1079/2174] 10.222.0.48 () {58 vars in 1058 bytes} [Fri Sep 9 21:06:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:49.663838 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a7301-79de-4460-9f3a-c53a4ea8b076 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:49.664980 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb292c-350b-4ae2-8165-90d2cefaac0f None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:49.669135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-50bb292c-350b-4ae2-8165-90d2cefaac0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:49.683806 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a7301-79de-4460-9f3a-c53a4ea8b076 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:49.688900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759b205-cb17-483f-8c6f-45a50e89fcb2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4f6de2065d4fbea3fa141da760f708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:49.689503 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759b205-cb17-483f-8c6f-45a50e89fcb2 service placement] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:49.689785 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759b205-cb17-483f-8c6f-45a50e89fcb2 service placement] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:49.690064 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759b205-cb17-483f-8c6f-45a50e89fcb2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:49.699730 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759b205-cb17-483f-8c6f-45a50e89fcb2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:49.701198 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759b205-cb17-483f-8c6f-45a50e89fcb2 service placement] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:49.731643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1096/2175] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:06:49.746951 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb292c-350b-4ae2-8165-90d2cefaac0f None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:49.753218 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e6a8e1-bce3-4260-927e-889ab96c346c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:49.754343 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6a8e1-bce3-4260-927e-889ab96c346c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:49.754849 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6a8e1-bce3-4260-927e-889ab96c346c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:49.755475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6a8e1-bce3-4260-927e-889ab96c346c admin admin] PATH_INFO: `/v3/users/83e9308215e94c5e9e615be1f2a113b1` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:49.757011 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e6a8e1-bce3-4260-927e-889ab96c346c admin admin] RBAC: Authorizing `identity:delete_user(user_id=83e9308215e94c5e9e615be1f2a113b1)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:49.762951 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e6a8e1-bce3-4260-927e-889ab96c346c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:49.910584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759b205-cb17-483f-8c6f-45a50e89fcb2 service placement] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:49.920813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759b205-cb17-483f-8c6f-45a50e89fcb2 service placement] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:49.924676 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e826c02-e51b-4d36-b368-2d4e7c3ebfa4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:49.925357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e826c02-e51b-4d36-b368-2d4e7c3ebfa4 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:49.925700 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e826c02-e51b-4d36-b368-2d4e7c3ebfa4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:49.925979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e826c02-e51b-4d36-b368-2d4e7c3ebfa4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:49.928393 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3e826c02-e51b-4d36-b368-2d4e7c3ebfa4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:50.012773 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e826c02-e51b-4d36-b368-2d4e7c3ebfa4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:50.012773 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e826c02-e51b-4d36-b368-2d4e7c3ebfa4 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:50.035732 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1097/2176] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:06:50.047194 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:50.053572 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-69e6a8e1-bce3-4260-927e-889ab96c346c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83e9308215e94c5e9e615be1f2a113b1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:50.054276 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-69e6a8e1-bce3-4260-927e-889ab96c346c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83e9308215e94c5e9e615be1f2a113b1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:50.068625 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-69e6a8e1-bce3-4260-927e-889ab96c346c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83e9308215e94c5e9e615be1f2a113b1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:50.091144 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-69e6a8e1-bce3-4260-927e-889ab96c346c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83e9308215e94c5e9e615be1f2a113b1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:50.091976 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-69e6a8e1-bce3-4260-927e-889ab96c346c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83e9308215e94c5e9e615be1f2a113b1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:50.192244 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1080/2177] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:06:49 2022] DELETE /identity/v3/users/83e9308215e94c5e9e615be1f2a113b1 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:50.471273 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e826c02-e51b-4d36-b368-2d4e7c3ebfa4 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:50.483464 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e826c02-e51b-4d36-b368-2d4e7c3ebfa4 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:50.498551 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:50.554710 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4dd676-9242-4207-83ac-c59ed4fad9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:50.555308 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4dd676-9242-4207-83ac-c59ed4fad9a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:50.555522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4dd676-9242-4207-83ac-c59ed4fad9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:50.555792 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4dd676-9242-4207-83ac-c59ed4fad9a7 admin admin] PATH_INFO: `/v3/projects/46978366f11c4af79a9e9ad6ff2e5e05` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:50.556952 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4dd676-9242-4207-83ac-c59ed4fad9a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46978366f11c4af79a9e9ad6ff2e5e05)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:50.558851 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4dd676-9242-4207-83ac-c59ed4fad9a7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:50.690228 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2a4dd676-9242-4207-83ac-c59ed4fad9a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46978366f11c4af79a9e9ad6ff2e5e05', 'request_id': 'req-2a4dd676-9242-4207-83ac-c59ed4fad9a7'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:50.697654 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2a4dd676-9242-4207-83ac-c59ed4fad9a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46978366f11c4af79a9e9ad6ff2e5e05', 'request_id': 'req-2a4dd676-9242-4207-83ac-c59ed4fad9a7'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:50.812453 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2a4dd676-9242-4207-83ac-c59ed4fad9a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46978366f11c4af79a9e9ad6ff2e5e05', 'request_id': 'req-2a4dd676-9242-4207-83ac-c59ed4fad9a7'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:50.824302 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2a4dd676-9242-4207-83ac-c59ed4fad9a7 admin admin] The token cache is being invalidate because project 46978366f11c4af79a9e9ad6ff2e5e05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:06:50.824588 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2a4dd676-9242-4207-83ac-c59ed4fad9a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:50.825202 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2a4dd676-9242-4207-83ac-c59ed4fad9a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:50.826662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1098/2178] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:06:50 2022] DELETE /identity/v3/projects/46978366f11c4af79a9e9ad6ff2e5e05 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:50.831782 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ff5b8-1c33-4422-8850-4ac867057656 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:50.832116 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ff5b8-1c33-4422-8850-4ac867057656 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:50.832396 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ff5b8-1c33-4422-8850-4ac867057656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:50.854303 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-4a1ff5b8-1c33-4422-8850-4ac867057656 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:50.855502 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-4a1ff5b8-1c33-4422-8850-4ac867057656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:50.916052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1081/2179] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:50.923597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba4da3-ed02-4c6e-be23-e07c8ce75cd2 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:50.923883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba4da3-ed02-4c6e-be23-e07c8ce75cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:50.924199 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba4da3-ed02-4c6e-be23-e07c8ce75cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:50.945826 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-12ba4da3-ed02-4c6e-be23-e07c8ce75cd2 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:50.947005 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-12ba4da3-ed02-4c6e-be23-e07c8ce75cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:50.955302 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1099/2180] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:50.960242 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ff5b8-1c33-4422-8850-4ac867057656 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:50.970115 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ff5b8-1c33-4422-8850-4ac867057656 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:50.974719 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c885f-871d-4be6-ae21-0f7f907d9475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:50.975313 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c885f-871d-4be6-ae21-0f7f907d9475 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:50.975596 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c885f-871d-4be6-ae21-0f7f907d9475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:50.975869 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c885f-871d-4be6-ae21-0f7f907d9475 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:50.976605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059c885f-871d-4be6-ae21-0f7f907d9475 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:06:50.976907 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059c885f-871d-4be6-ae21-0f7f907d9475 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:50.978510 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059c885f-871d-4be6-ae21-0f7f907d9475 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:50.991876 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1082/2181] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:06:50.996296 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba4da3-ed02-4c6e-be23-e07c8ce75cd2 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:51.006191 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba4da3-ed02-4c6e-be23-e07c8ce75cd2 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:51.010967 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b33ec5-702a-41ea-a1e6-80544a197ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:51.011562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b33ec5-702a-41ea-a1e6-80544a197ac2 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:51.011917 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b33ec5-702a-41ea-a1e6-80544a197ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:51.012228 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b33ec5-702a-41ea-a1e6-80544a197ac2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:51.012880 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b33ec5-702a-41ea-a1e6-80544a197ac2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:51.014791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b33ec5-702a-41ea-a1e6-80544a197ac2 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:51.126273 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1100/2182] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:06:50 2022] POST /identity/v3/projects => generated 405 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:51.131068 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c885f-871d-4be6-ae21-0f7f907d9475 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:51.141036 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c885f-871d-4be6-ae21-0f7f907d9475 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:51.198726 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c09bdc-6262-4cc1-a72a-f74017baf7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:51.199617 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c09bdc-6262-4cc1-a72a-f74017baf7a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:51.199959 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c09bdc-6262-4cc1-a72a-f74017baf7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:51.200265 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c09bdc-6262-4cc1-a72a-f74017baf7a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:51.204947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c09bdc-6262-4cc1-a72a-f74017baf7a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:51.204947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c09bdc-6262-4cc1-a72a-f74017baf7a8 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:51.298658 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1083/2183] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:06:51 2022] POST /identity/v3/users => generated 303 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:51.303470 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b33ec5-702a-41ea-a1e6-80544a197ac2 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:51.314279 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b33ec5-702a-41ea-a1e6-80544a197ac2 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:51.319481 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd898a-77a3-49c8-8506-94d879ce7eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:51.320124 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd898a-77a3-49c8-8506-94d879ce7eab admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:51.320413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd898a-77a3-49c8-8506-94d879ce7eab admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:51.320685 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd898a-77a3-49c8-8506-94d879ce7eab admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:51.321336 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccd898a-77a3-49c8-8506-94d879ce7eab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:51.322944 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccd898a-77a3-49c8-8506-94d879ce7eab admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:51.333425 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1101/2184] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:51.337680 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c09bdc-6262-4cc1-a72a-f74017baf7a8 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:51.347832 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c09bdc-6262-4cc1-a72a-f74017baf7a8 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:51.353443 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c2040-6a12-4c36-b51a-7ebda825e878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:51.354237 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c2040-6a12-4c36-b51a-7ebda825e878 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:51.354358 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c2040-6a12-4c36-b51a-7ebda825e878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:51.354556 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c2040-6a12-4c36-b51a-7ebda825e878 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:51.355203 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c2040-6a12-4c36-b51a-7ebda825e878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:51.356829 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c2040-6a12-4c36-b51a-7ebda825e878 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:51.369094 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1084/2185] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:51.376421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd898a-77a3-49c8-8506-94d879ce7eab admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:51.386340 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd898a-77a3-49c8-8506-94d879ce7eab admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:51.390957 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82cc60-e15b-45f7-9b83-8e6c41066f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:51.391599 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82cc60-e15b-45f7-9b83-8e6c41066f21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:51.391883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82cc60-e15b-45f7-9b83-8e6c41066f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:51.392185 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82cc60-e15b-45f7-9b83-8e6c41066f21 admin admin] PATH_INFO: `/v3/projects/f007858477d649fbb7c2588bddcbab64/users/8d01b0482884435c85e6f634b0de3a74/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:51.406358 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae82cc60-e15b-45f7-9b83-8e6c41066f21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f007858477d649fbb7c2588bddcbab64, user_id=8d01b0482884435c85e6f634b0de3a74, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:51.409077 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae82cc60-e15b-45f7-9b83-8e6c41066f21 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:51.429795 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:51.486035 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1102/2186] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:51 2022] PUT /identity/v3/projects/f007858477d649fbb7c2588bddcbab64/users/8d01b0482884435c85e6f634b0de3a74/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:51.494281 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11cdb4-8120-4ff8-a733-a9c020f355cc None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:51.494281 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11cdb4-8120-4ff8-a733-a9c020f355cc None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:51.494281 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11cdb4-8120-4ff8-a733-a9c020f355cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:51.511857 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-8c11cdb4-8120-4ff8-a733-a9c020f355cc None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:51.513086 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8c11cdb4-8120-4ff8-a733-a9c020f355cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:51.574246 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1085/2187] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:51.584774 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82cc60-e15b-45f7-9b83-8e6c41066f21 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:51.586482 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ae82cc60-e15b-45f7-9b83-8e6c41066f21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:51.644423 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82cc60-e15b-45f7-9b83-8e6c41066f21 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:51.648544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08884ab-c0db-49df-a3a1-cb05dc8381d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:51.649256 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08884ab-c0db-49df-a3a1-cb05dc8381d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:51.649536 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08884ab-c0db-49df-a3a1-cb05dc8381d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:51.649809 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08884ab-c0db-49df-a3a1-cb05dc8381d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:51.659173 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08884ab-c0db-49df-a3a1-cb05dc8381d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:51.660925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08884ab-c0db-49df-a3a1-cb05dc8381d2 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:51.671125 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1103/2188] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:06:51.801403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:52.545449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:53.163162 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1df02-8850-4306-9c94-bfce22fd0c7a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:53.163785 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1df02-8850-4306-9c94-bfce22fd0c7a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:53.164275 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1df02-8850-4306-9c94-bfce22fd0c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:53.227800 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-15b1df02-8850-4306-9c94-bfce22fd0c7a None None] MFA Rules not processed for user `8d01b0482884435c85e6f634b0de3a74`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:53.229696 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-15b1df02-8850-4306-9c94-bfce22fd0c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:53.237545 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:53.316951 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1086/2189] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:53 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:53.322740 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08884ab-c0db-49df-a3a1-cb05dc8381d2 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:53.334691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08884ab-c0db-49df-a3a1-cb05dc8381d2 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:53.339873 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330eaacf-6822-4224-a5b6-b21b95abbb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:53.340503 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330eaacf-6822-4224-a5b6-b21b95abbb01 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:53.340782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330eaacf-6822-4224-a5b6-b21b95abbb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:53.341193 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330eaacf-6822-4224-a5b6-b21b95abbb01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:53.342636 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330eaacf-6822-4224-a5b6-b21b95abbb01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:53.345773 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330eaacf-6822-4224-a5b6-b21b95abbb01 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:53.454071 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1104/2190] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:06:53 2022] POST /identity/v3/projects => generated 403 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:53.457958 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b1df02-8850-4306-9c94-bfce22fd0c7a None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:53.468966 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b1df02-8850-4306-9c94-bfce22fd0c7a None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:53.532846 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc5c5c-ddab-4ada-90d3-5dce53194f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:53.533457 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc5c5c-ddab-4ada-90d3-5dce53194f6c admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:53.533733 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc5c5c-ddab-4ada-90d3-5dce53194f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:53.534003 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc5c5c-ddab-4ada-90d3-5dce53194f6c admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:53.534631 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc5c5c-ddab-4ada-90d3-5dce53194f6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:53.537087 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc5c5c-ddab-4ada-90d3-5dce53194f6c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:53.646500 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1087/2191] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:06:53 2022] POST /identity/v3/users => generated 308 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:53.650684 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330eaacf-6822-4224-a5b6-b21b95abbb01 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:53.661986 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330eaacf-6822-4224-a5b6-b21b95abbb01 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:53.666909 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090d2d9-a2fb-452b-8579-6e64408d42b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:53.667471 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090d2d9-a2fb-452b-8579-6e64408d42b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:53.667756 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090d2d9-a2fb-452b-8579-6e64408d42b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:53.668237 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090d2d9-a2fb-452b-8579-6e64408d42b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:53.673569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9090d2d9-a2fb-452b-8579-6e64408d42b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:53.673569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9090d2d9-a2fb-452b-8579-6e64408d42b3 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:53.685695 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1105/2192] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:53.689686 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc5c5c-ddab-4ada-90d3-5dce53194f6c admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:53.700266 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc5c5c-ddab-4ada-90d3-5dce53194f6c admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:53.705360 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1db1d-2579-42c1-8971-14518a2dac0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:53.705945 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1db1d-2579-42c1-8971-14518a2dac0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:53.706224 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1db1d-2579-42c1-8971-14518a2dac0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:53.706496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1db1d-2579-42c1-8971-14518a2dac0e admin admin] PATH_INFO: `/v3/projects/d13cf911fa9e4df595722b2cbd42e23f/users/84cb6ba9fb8a4e05975b2341e8e1063c/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:53.721803 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1db1d-2579-42c1-8971-14518a2dac0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d13cf911fa9e4df595722b2cbd42e23f, user_id=84cb6ba9fb8a4e05975b2341e8e1063c, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:53.725049 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1db1d-2579-42c1-8971-14518a2dac0e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:53.841573 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1088/2193] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:53 2022] PUT /identity/v3/projects/d13cf911fa9e4df595722b2cbd42e23f/users/84cb6ba9fb8a4e05975b2341e8e1063c/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:53.845658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090d2d9-a2fb-452b-8579-6e64408d42b3 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:53.856333 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090d2d9-a2fb-452b-8579-6e64408d42b3 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:53.922685 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7c0dd-e5cb-4bc6-8356-1cabf43fe85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:53.923352 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7c0dd-e5cb-4bc6-8356-1cabf43fe85c admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:53.923629 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7c0dd-e5cb-4bc6-8356-1cabf43fe85c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:53.923900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7c0dd-e5cb-4bc6-8356-1cabf43fe85c admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:53.924591 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df7c0dd-e5cb-4bc6-8356-1cabf43fe85c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:53.926309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df7c0dd-e5cb-4bc6-8356-1cabf43fe85c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:53.936834 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1106/2194] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:53.941052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1db1d-2579-42c1-8971-14518a2dac0e admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:53.952221 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1db1d-2579-42c1-8971-14518a2dac0e admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:53.957096 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f95b230-ec22-4c43-9c3a-f6ebeb647a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:53.957744 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95b230-ec22-4c43-9c3a-f6ebeb647a66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:53.958048 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95b230-ec22-4c43-9c3a-f6ebeb647a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:53.958855 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95b230-ec22-4c43-9c3a-f6ebeb647a66 admin admin] PATH_INFO: `/v3/projects/d13cf911fa9e4df595722b2cbd42e23f/users/84cb6ba9fb8a4e05975b2341e8e1063c/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:53.961039 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f95b230-ec22-4c43-9c3a-f6ebeb647a66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d13cf911fa9e4df595722b2cbd42e23f, user_id=84cb6ba9fb8a4e05975b2341e8e1063c, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:53.963728 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f95b230-ec22-4c43-9c3a-f6ebeb647a66 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:53.974653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1089/2195] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:53 2022] PUT /identity/v3/projects/d13cf911fa9e4df595722b2cbd42e23f/users/84cb6ba9fb8a4e05975b2341e8e1063c/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:06:54.085938 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:54.895404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:55.322361 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2366a-ce37-4161-afd0-a1354ca766b9 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:55.322842 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2366a-ce37-4161-afd0-a1354ca766b9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:55.323201 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2366a-ce37-4161-afd0-a1354ca766b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:55.363503 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-20f2366a-ce37-4161-afd0-a1354ca766b9 None None] MFA Rules not processed for user `84cb6ba9fb8a4e05975b2341e8e1063c`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:55.365003 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-20f2366a-ce37-4161-afd0-a1354ca766b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:55.445396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1107/2196] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:55 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:55.738136 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:56.207041 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:56.802949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633af0b-e7bf-4e4c-ba46-b99f1cfbe534 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:56.803974 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633af0b-e7bf-4e4c-ba46-b99f1cfbe534 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:56.804302 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633af0b-e7bf-4e4c-ba46-b99f1cfbe534 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:56.805757 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1090/2197] 10.222.0.48 () {54 vars in 921 bytes} [Fri Sep 9 21:06:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:06:56.837765 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227bf8c9-b5de-4e71-bb70-250cba47824e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:56.838237 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227bf8c9-b5de-4e71-bb70-250cba47824e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:56.838439 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227bf8c9-b5de-4e71-bb70-250cba47824e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:56.859168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-227bf8c9-b5de-4e71-bb70-250cba47824e None None] MFA Rules not processed for user `2a4f6de2065d4fbea3fa141da760f708`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:56.860531 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-227bf8c9-b5de-4e71-bb70-250cba47824e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:56.939257 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1108/2198] 10.222.0.48 () {58 vars in 1018 bytes} [Fri Sep 9 21:06:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:56.953030 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a633af0b-e7bf-4e4c-ba46-b99f1cfbe534 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:56.955469 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a633af0b-e7bf-4e4c-ba46-b99f1cfbe534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:56.971074 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227bf8c9-b5de-4e71-bb70-250cba47824e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:56.988975 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a633af0b-e7bf-4e4c-ba46-b99f1cfbe534 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:56.994422 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d645f536-b368-4da0-89fb-10d96f261283 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4f6de2065d4fbea3fa141da760f708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:56.995039 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645f536-b368-4da0-89fb-10d96f261283 service placement] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:56.995327 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645f536-b368-4da0-89fb-10d96f261283 service placement] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:56.995604 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645f536-b368-4da0-89fb-10d96f261283 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:57.005375 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d645f536-b368-4da0-89fb-10d96f261283 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:57.007094 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d645f536-b368-4da0-89fb-10d96f261283 service placement] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:57.019621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1091/2199] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:06:57.048106 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227bf8c9-b5de-4e71-bb70-250cba47824e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:57.052958 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b6eb0-7f00-4195-9e80-61ba8a2341ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:57.053748 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b6eb0-7f00-4195-9e80-61ba8a2341ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:57.054031 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b6eb0-7f00-4195-9e80-61ba8a2341ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:57.054305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b6eb0-7f00-4195-9e80-61ba8a2341ae admin admin] PATH_INFO: `/v3/users/8d01b0482884435c85e6f634b0de3a74` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:57.055522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b6eb0-7f00-4195-9e80-61ba8a2341ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d01b0482884435c85e6f634b0de3a74)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:57.057783 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b6eb0-7f00-4195-9e80-61ba8a2341ae admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:57.172347 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d645f536-b368-4da0-89fb-10d96f261283 service placement] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:57.173797 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d645f536-b368-4da0-89fb-10d96f261283 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:57.187738 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d645f536-b368-4da0-89fb-10d96f261283 service placement] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:57.196102 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96bdc8-4ec6-4522-826a-e2bb2d9f089a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:57.196751 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96bdc8-4ec6-4522-826a-e2bb2d9f089a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:57.197029 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96bdc8-4ec6-4522-826a-e2bb2d9f089a admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:57.197359 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96bdc8-4ec6-4522-826a-e2bb2d9f089a admin admin] PATH_INFO: `/v3/users/f22db2ce76984ba3a7013439fa053018` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:57.198493 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d96bdc8-4ec6-4522-826a-e2bb2d9f089a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f22db2ce76984ba3a7013439fa053018)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:57.200446 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d96bdc8-4ec6-4522-826a-e2bb2d9f089a admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:57.214492 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:57.256205 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-590b6eb0-7f00-4195-9e80-61ba8a2341ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d01b0482884435c85e6f634b0de3a74'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:57.256897 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-590b6eb0-7f00-4195-9e80-61ba8a2341ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d01b0482884435c85e6f634b0de3a74'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:57.257510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-590b6eb0-7f00-4195-9e80-61ba8a2341ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d01b0482884435c85e6f634b0de3a74'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:57.264503 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-590b6eb0-7f00-4195-9e80-61ba8a2341ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d01b0482884435c85e6f634b0de3a74'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:57.278514 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-590b6eb0-7f00-4195-9e80-61ba8a2341ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d01b0482884435c85e6f634b0de3a74'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:57.406492 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1109/2200] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:06:56 2022] DELETE /identity/v3/users/8d01b0482884435c85e6f634b0de3a74 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:57.446305 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5d96bdc8-4ec6-4522-826a-e2bb2d9f089a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f22db2ce76984ba3a7013439fa053018'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:57.446977 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5d96bdc8-4ec6-4522-826a-e2bb2d9f089a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f22db2ce76984ba3a7013439fa053018'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:57.454790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5d96bdc8-4ec6-4522-826a-e2bb2d9f089a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f22db2ce76984ba3a7013439fa053018'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:57.468556 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5d96bdc8-4ec6-4522-826a-e2bb2d9f089a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f22db2ce76984ba3a7013439fa053018'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:57.469301 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5d96bdc8-4ec6-4522-826a-e2bb2d9f089a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f22db2ce76984ba3a7013439fa053018'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:57.546177 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1092/2201] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:06:57 2022] DELETE /identity/v3/users/f22db2ce76984ba3a7013439fa053018 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:06:57.548878 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180c6e4-b7af-4803-8e8c-0dd432aaccb1 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:57.549171 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180c6e4-b7af-4803-8e8c-0dd432aaccb1 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:57.549959 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180c6e4-b7af-4803-8e8c-0dd432aaccb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:57.572351 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a180c6e4-b7af-4803-8e8c-0dd432aaccb1 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:57.573805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a180c6e4-b7af-4803-8e8c-0dd432aaccb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:57.652979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1110/2202] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:57.662702 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96bdc8-4ec6-4522-826a-e2bb2d9f089a admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:57.666702 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5d96bdc8-4ec6-4522-826a-e2bb2d9f089a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:57.695506 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180c6e4-b7af-4803-8e8c-0dd432aaccb1 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:57.695636 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a180c6e4-b7af-4803-8e8c-0dd432aaccb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:57.709963 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180c6e4-b7af-4803-8e8c-0dd432aaccb1 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:57.714897 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead7384-18cd-4c79-a359-3481de6a0303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:57.715743 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead7384-18cd-4c79-a359-3481de6a0303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:57.716243 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead7384-18cd-4c79-a359-3481de6a0303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:57.716658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead7384-18cd-4c79-a359-3481de6a0303 admin admin] PATH_INFO: `/v3/projects/f007858477d649fbb7c2588bddcbab64` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:57.718108 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead7384-18cd-4c79-a359-3481de6a0303 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f007858477d649fbb7c2588bddcbab64)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:57.721753 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead7384-18cd-4c79-a359-3481de6a0303 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:57.729038 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96bdc8-4ec6-4522-826a-e2bb2d9f089a admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:57.733572 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a47ddb-18d6-4da2-8c94-c3abe8ae352e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:57.734197 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a47ddb-18d6-4da2-8c94-c3abe8ae352e service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:57.734474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a47ddb-18d6-4da2-8c94-c3abe8ae352e service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:57.734748 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a47ddb-18d6-4da2-8c94-c3abe8ae352e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:57.745139 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a47ddb-18d6-4da2-8c94-c3abe8ae352e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:57.747302 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a47ddb-18d6-4da2-8c94-c3abe8ae352e service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:57.760060 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1093/2203] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:06:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:06:57.876982 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fead7384-18cd-4c79-a359-3481de6a0303 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f007858477d649fbb7c2588bddcbab64', 'request_id': 'req-fead7384-18cd-4c79-a359-3481de6a0303'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:57.883969 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fead7384-18cd-4c79-a359-3481de6a0303 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f007858477d649fbb7c2588bddcbab64', 'request_id': 'req-fead7384-18cd-4c79-a359-3481de6a0303'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:57.984310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fead7384-18cd-4c79-a359-3481de6a0303 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f007858477d649fbb7c2588bddcbab64', 'request_id': 'req-fead7384-18cd-4c79-a359-3481de6a0303'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:57.999736 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fead7384-18cd-4c79-a359-3481de6a0303 admin admin] The token cache is being invalidate because project f007858477d649fbb7c2588bddcbab64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:06:57.999854 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fead7384-18cd-4c79-a359-3481de6a0303 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:58.000575 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fead7384-18cd-4c79-a359-3481de6a0303 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:58.002350 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1111/2204] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:06:57 2022] DELETE /identity/v3/projects/f007858477d649fbb7c2588bddcbab64 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:06:58.006328 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a47ddb-18d6-4da2-8c94-c3abe8ae352e service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:58.019669 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a47ddb-18d6-4da2-8c94-c3abe8ae352e service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:58.087457 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f3c49-1cd4-4786-ae09-080c9e04f0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:58.088145 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f3c49-1cd4-4786-ae09-080c9e04f0fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:58.089000 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f3c49-1cd4-4786-ae09-080c9e04f0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:58.089000 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f3c49-1cd4-4786-ae09-080c9e04f0fb admin admin] PATH_INFO: `/v3/users/84cb6ba9fb8a4e05975b2341e8e1063c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:58.090905 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f3c49-1cd4-4786-ae09-080c9e04f0fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=84cb6ba9fb8a4e05975b2341e8e1063c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:58.093029 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f3c49-1cd4-4786-ae09-080c9e04f0fb admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:58.281502 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ec2f3c49-1cd4-4786-ae09-080c9e04f0fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84cb6ba9fb8a4e05975b2341e8e1063c'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:58.282209 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ec2f3c49-1cd4-4786-ae09-080c9e04f0fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84cb6ba9fb8a4e05975b2341e8e1063c'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:58.289987 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ec2f3c49-1cd4-4786-ae09-080c9e04f0fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84cb6ba9fb8a4e05975b2341e8e1063c'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:58.304542 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:58.312015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead7384-18cd-4c79-a359-3481de6a0303 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:58.312371 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ec2f3c49-1cd4-4786-ae09-080c9e04f0fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84cb6ba9fb8a4e05975b2341e8e1063c'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:58.314835 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ec2f3c49-1cd4-4786-ae09-080c9e04f0fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84cb6ba9fb8a4e05975b2341e8e1063c'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:58.314956 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fead7384-18cd-4c79-a359-3481de6a0303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:58.329702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead7384-18cd-4c79-a359-3481de6a0303 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:58.335797 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08425eb7-150a-4c63-85ea-38273dc79e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:58.336606 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08425eb7-150a-4c63-85ea-38273dc79e4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:58.336983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08425eb7-150a-4c63-85ea-38273dc79e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:58.337311 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08425eb7-150a-4c63-85ea-38273dc79e4f admin admin] PATH_INFO: `/v3/projects/ce69fa81b3ff401ab7cbeafdb0bdfa11` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:58.338633 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08425eb7-150a-4c63-85ea-38273dc79e4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce69fa81b3ff401ab7cbeafdb0bdfa11)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:58.341306 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08425eb7-150a-4c63-85ea-38273dc79e4f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:58.394040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1094/2205] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:06:58 2022] DELETE /identity/v3/users/84cb6ba9fb8a4e05975b2341e8e1063c => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:58.448135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-08425eb7-150a-4c63-85ea-38273dc79e4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce69fa81b3ff401ab7cbeafdb0bdfa11', 'request_id': 'req-08425eb7-150a-4c63-85ea-38273dc79e4f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:58.455395 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-08425eb7-150a-4c63-85ea-38273dc79e4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce69fa81b3ff401ab7cbeafdb0bdfa11', 'request_id': 'req-08425eb7-150a-4c63-85ea-38273dc79e4f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:58.497894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-08425eb7-150a-4c63-85ea-38273dc79e4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce69fa81b3ff401ab7cbeafdb0bdfa11', 'request_id': 'req-08425eb7-150a-4c63-85ea-38273dc79e4f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:58.509325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-08425eb7-150a-4c63-85ea-38273dc79e4f admin admin] The token cache is being invalidate because project ce69fa81b3ff401ab7cbeafdb0bdfa11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:06:58.509620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-08425eb7-150a-4c63-85ea-38273dc79e4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:58.510277 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-08425eb7-150a-4c63-85ea-38273dc79e4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:58.511771 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1112/2206] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:06:58 2022] DELETE /identity/v3/projects/ce69fa81b3ff401ab7cbeafdb0bdfa11 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:06:58.515942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f3c49-1cd4-4786-ae09-080c9e04f0fb admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:58.517546 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ec2f3c49-1cd4-4786-ae09-080c9e04f0fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:58.590388 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f3c49-1cd4-4786-ae09-080c9e04f0fb admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:58.645114 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf76444-c7cc-4223-b2d2-e15458cea70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:58.646261 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf76444-c7cc-4223-b2d2-e15458cea70d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:58.646518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf76444-c7cc-4223-b2d2-e15458cea70d admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:58.646786 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf76444-c7cc-4223-b2d2-e15458cea70d admin admin] PATH_INFO: `/v3/users/ca00ba6208e941638da71ec4dba57e71` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:58.647924 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf76444-c7cc-4223-b2d2-e15458cea70d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca00ba6208e941638da71ec4dba57e71)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:58.649895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf76444-c7cc-4223-b2d2-e15458cea70d admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:58.659475 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08425eb7-150a-4c63-85ea-38273dc79e4f admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:58.661039 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-08425eb7-150a-4c63-85ea-38273dc79e4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:58.688175 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08425eb7-150a-4c63-85ea-38273dc79e4f admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:58.693673 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36472bb4-7bec-4f4f-9a64-f011cac1b374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:58.694260 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36472bb4-7bec-4f4f-9a64-f011cac1b374 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:58.694518 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36472bb4-7bec-4f4f-9a64-f011cac1b374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:58.694782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36472bb4-7bec-4f4f-9a64-f011cac1b374 admin admin] PATH_INFO: `/v3/projects/d13cf911fa9e4df595722b2cbd42e23f` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:58.695892 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36472bb4-7bec-4f4f-9a64-f011cac1b374 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d13cf911fa9e4df595722b2cbd42e23f)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:58.698002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36472bb4-7bec-4f4f-9a64-f011cac1b374 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:58.884267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-cbf76444-c7cc-4223-b2d2-e15458cea70d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca00ba6208e941638da71ec4dba57e71'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:58.885149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-36472bb4-7bec-4f4f-9a64-f011cac1b374 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd13cf911fa9e4df595722b2cbd42e23f', 'request_id': 'req-36472bb4-7bec-4f4f-9a64-f011cac1b374'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:58.885631 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-cbf76444-c7cc-4223-b2d2-e15458cea70d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca00ba6208e941638da71ec4dba57e71'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:58.892162 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-36472bb4-7bec-4f4f-9a64-f011cac1b374 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd13cf911fa9e4df595722b2cbd42e23f', 'request_id': 'req-36472bb4-7bec-4f4f-9a64-f011cac1b374'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:58.893001 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-cbf76444-c7cc-4223-b2d2-e15458cea70d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca00ba6208e941638da71ec4dba57e71'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:58.899767 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:58.907908 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-cbf76444-c7cc-4223-b2d2-e15458cea70d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca00ba6208e941638da71ec4dba57e71'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:58.908752 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-cbf76444-c7cc-4223-b2d2-e15458cea70d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca00ba6208e941638da71ec4dba57e71'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:58.994952 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-36472bb4-7bec-4f4f-9a64-f011cac1b374 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd13cf911fa9e4df595722b2cbd42e23f', 'request_id': 'req-36472bb4-7bec-4f4f-9a64-f011cac1b374'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:59.006621 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-36472bb4-7bec-4f4f-9a64-f011cac1b374 admin admin] The token cache is being invalidate because project d13cf911fa9e4df595722b2cbd42e23f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:06:59.006911 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-36472bb4-7bec-4f4f-9a64-f011cac1b374 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:59.007580 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-36472bb4-7bec-4f4f-9a64-f011cac1b374 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:06:59.009105 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1113/2207] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:06:58 2022] DELETE /identity/v3/projects/d13cf911fa9e4df595722b2cbd42e23f => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:06:59.014585 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab55c9-dd8c-4ded-9bb5-afcca4385cce None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:59.016431 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab55c9-dd8c-4ded-9bb5-afcca4385cce None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:59.016431 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab55c9-dd8c-4ded-9bb5-afcca4385cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:59.034996 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-22ab55c9-dd8c-4ded-9bb5-afcca4385cce None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:59.036458 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-22ab55c9-dd8c-4ded-9bb5-afcca4385cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:59.087178 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1095/2208] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:06:58 2022] DELETE /identity/v3/users/ca00ba6208e941638da71ec4dba57e71 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:59.096341 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1114/2209] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:06:59.116008 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2d08d-29c6-49b9-85ef-9820966ff5a0 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:59.116333 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2d08d-29c6-49b9-85ef-9820966ff5a0 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:59.116984 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2d08d-29c6-49b9-85ef-9820966ff5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:59.151287 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-abe2d08d-29c6-49b9-85ef-9820966ff5a0 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:59.152689 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-abe2d08d-29c6-49b9-85ef-9820966ff5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:59.229448 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1096/2210] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:06:59.234867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab55c9-dd8c-4ded-9bb5-afcca4385cce None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:59.247709 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab55c9-dd8c-4ded-9bb5-afcca4385cce None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:59.256458 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06733f50-0588-4963-9943-384a9cba329a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:59.257079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06733f50-0588-4963-9943-384a9cba329a admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:59.257357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06733f50-0588-4963-9943-384a9cba329a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:59.257616 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06733f50-0588-4963-9943-384a9cba329a admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:59.258263 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06733f50-0588-4963-9943-384a9cba329a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:06:59.258564 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06733f50-0588-4963-9943-384a9cba329a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:59.260263 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06733f50-0588-4963-9943-384a9cba329a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:59.275405 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1115/2211] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:06:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:06:59.282296 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2d08d-29c6-49b9-85ef-9820966ff5a0 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:59.292684 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2d08d-29c6-49b9-85ef-9820966ff5a0 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:59.298691 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08a420-9d66-4af4-93b8-f0b1f47e0479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:59.298691 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08a420-9d66-4af4-93b8-f0b1f47e0479 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:59.298691 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08a420-9d66-4af4-93b8-f0b1f47e0479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:59.298964 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08a420-9d66-4af4-93b8-f0b1f47e0479 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:59.299640 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f08a420-9d66-4af4-93b8-f0b1f47e0479 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:59.305496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f08a420-9d66-4af4-93b8-f0b1f47e0479 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:59.422554 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1097/2212] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:06:59 2022] POST /identity/v3/projects => generated 357 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:59.427512 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06733f50-0588-4963-9943-384a9cba329a admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:59.439420 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06733f50-0588-4963-9943-384a9cba329a admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:59.522160 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f539140-81eb-418c-9b25-de567f7a23d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:59.523188 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f539140-81eb-418c-9b25-de567f7a23d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:59.523188 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f539140-81eb-418c-9b25-de567f7a23d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:59.523398 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f539140-81eb-418c-9b25-de567f7a23d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:59.524290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f539140-81eb-418c-9b25-de567f7a23d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:59.526315 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f539140-81eb-418c-9b25-de567f7a23d6 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:59.638747 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1116/2213] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:06:59 2022] POST /identity/v3/users => generated 279 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:06:59.640048 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08a420-9d66-4af4-93b8-f0b1f47e0479 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:59.653518 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:06:59.670303 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08a420-9d66-4af4-93b8-f0b1f47e0479 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:59.675818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e5544-a85f-498a-af35-156015e0bd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:59.676494 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e5544-a85f-498a-af35-156015e0bd76 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:59.677406 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e5544-a85f-498a-af35-156015e0bd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:59.677636 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e5544-a85f-498a-af35-156015e0bd76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:59.678380 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e5544-a85f-498a-af35-156015e0bd76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:59.679957 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e5544-a85f-498a-af35-156015e0bd76 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:59.691623 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1098/2214] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:06:59.696187 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f539140-81eb-418c-9b25-de567f7a23d6 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:59.708838 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f539140-81eb-418c-9b25-de567f7a23d6 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:59.714042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c1d70-7935-43be-8ec3-5815f08557cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:59.714730 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c1d70-7935-43be-8ec3-5815f08557cb admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:59.715214 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c1d70-7935-43be-8ec3-5815f08557cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:59.715566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c1d70-7935-43be-8ec3-5815f08557cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:59.716490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c1d70-7935-43be-8ec3-5815f08557cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:59.718061 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c1d70-7935-43be-8ec3-5815f08557cb admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:59.730442 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1117/2215] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:06:59.736057 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e5544-a85f-498a-af35-156015e0bd76 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:59.747201 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e5544-a85f-498a-af35-156015e0bd76 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:59.752559 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40106a2-efb7-49a1-ab98-3fe84fa1e11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:59.753488 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40106a2-efb7-49a1-ab98-3fe84fa1e11b admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:59.753869 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40106a2-efb7-49a1-ab98-3fe84fa1e11b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:59.754153 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40106a2-efb7-49a1-ab98-3fe84fa1e11b admin admin] PATH_INFO: `/v3/projects/9ca33fbf52a14fe3a25b3c8b77946524/users/399677d2da32463fa2389b38a146068a/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:59.774218 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40106a2-efb7-49a1-ab98-3fe84fa1e11b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ca33fbf52a14fe3a25b3c8b77946524, user_id=399677d2da32463fa2389b38a146068a, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:59.776861 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40106a2-efb7-49a1-ab98-3fe84fa1e11b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:59.842835 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c1d70-7935-43be-8ec3-5815f08557cb admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:06:59.844692 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a70c1d70-7935-43be-8ec3-5815f08557cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:06:59.860775 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c1d70-7935-43be-8ec3-5815f08557cb admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:06:59.872308 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb893f0-e231-4172-82a8-be91084eab64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:06:59.872607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1099/2216] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:06:59 2022] PUT /identity/v3/projects/9ca33fbf52a14fe3a25b3c8b77946524/users/399677d2da32463fa2389b38a146068a/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:06:59.873480 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb893f0-e231-4172-82a8-be91084eab64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:59.873964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb893f0-e231-4172-82a8-be91084eab64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:59.874406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb893f0-e231-4172-82a8-be91084eab64 admin admin] PATH_INFO: `/v3/projects/828937e1a4fa4d0094ab782fea9c3e36` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:59.882891 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb893f0-e231-4172-82a8-be91084eab64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=828937e1a4fa4d0094ab782fea9c3e36)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:06:59.883880 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0fc81-b038-4541-abf5-5c77c57fd400 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:06:59.885258 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb893f0-e231-4172-82a8-be91084eab64 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:06:59.886148 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0fc81-b038-4541-abf5-5c77c57fd400 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:06:59.886678 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0fc81-b038-4541-abf5-5c77c57fd400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:06:59.927176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-cbb0fc81-b038-4541-abf5-5c77c57fd400 None None] MFA Rules not processed for user `399677d2da32463fa2389b38a146068a`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:06:59.928483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-cbb0fc81-b038-4541-abf5-5c77c57fd400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:00.001111 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7eb893f0-e231-4172-82a8-be91084eab64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '828937e1a4fa4d0094ab782fea9c3e36', 'request_id': 'req-7eb893f0-e231-4172-82a8-be91084eab64'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:00.015874 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1100/2217] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:06:59 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:07:00.017136 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7eb893f0-e231-4172-82a8-be91084eab64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '828937e1a4fa4d0094ab782fea9c3e36', 'request_id': 'req-7eb893f0-e231-4172-82a8-be91084eab64'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:00.025390 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0fc81-b038-4541-abf5-5c77c57fd400 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:00.039679 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:00.088961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0fc81-b038-4541-abf5-5c77c57fd400 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:00.103456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7eb893f0-e231-4172-82a8-be91084eab64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '828937e1a4fa4d0094ab782fea9c3e36', 'request_id': 'req-7eb893f0-e231-4172-82a8-be91084eab64'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:00.115078 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7eb893f0-e231-4172-82a8-be91084eab64 admin admin] The token cache is being invalidate because project 828937e1a4fa4d0094ab782fea9c3e36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:07:00.115374 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7eb893f0-e231-4172-82a8-be91084eab64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:00.116039 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7eb893f0-e231-4172-82a8-be91084eab64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:00.117745 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1118/2218] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:06:59 2022] DELETE /identity/v3/projects/828937e1a4fa4d0094ab782fea9c3e36 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:07:00.121632 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb893f0-e231-4172-82a8-be91084eab64 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:00.124243 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7eb893f0-e231-4172-82a8-be91084eab64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:00.154403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f0b55-c078-4f9f-b4e3-fca7b51bfc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:00.155033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f0b55-c078-4f9f-b4e3-fca7b51bfc08 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:00.155305 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f0b55-c078-4f9f-b4e3-fca7b51bfc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:00.155579 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f0b55-c078-4f9f-b4e3-fca7b51bfc08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:00.156264 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f0b55-c078-4f9f-b4e3-fca7b51bfc08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:00.158216 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f0b55-c078-4f9f-b4e3-fca7b51bfc08 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:00.195698 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:00.211787 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb893f0-e231-4172-82a8-be91084eab64 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:00.216589 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b751c9-55bd-4699-b275-74f81593d1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:00.217344 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b751c9-55bd-4699-b275-74f81593d1b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:00.217626 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b751c9-55bd-4699-b275-74f81593d1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:00.217902 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b751c9-55bd-4699-b275-74f81593d1b9 admin admin] PATH_INFO: `/v3/users/a53f72887f114989b8c013b87ac0be04` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:00.219044 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b751c9-55bd-4699-b275-74f81593d1b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a53f72887f114989b8c013b87ac0be04)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:00.221080 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b751c9-55bd-4699-b275-74f81593d1b9 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:00.253902 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1101/2219] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:07:00 2022] POST /identity/v3/projects => generated 357 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:07:00.261081 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f0b55-c078-4f9f-b4e3-fca7b51bfc08 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:00.262770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c90f0b55-c078-4f9f-b4e3-fca7b51bfc08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:00.330773 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f0b55-c078-4f9f-b4e3-fca7b51bfc08 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:00.336598 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99adadf6-35f1-4906-bb7a-2b81845cb38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:00.337214 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99adadf6-35f1-4906-bb7a-2b81845cb38d admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:00.337612 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99adadf6-35f1-4906-bb7a-2b81845cb38d admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:00.337895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99adadf6-35f1-4906-bb7a-2b81845cb38d admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:00.338535 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99adadf6-35f1-4906-bb7a-2b81845cb38d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:00.340560 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99adadf6-35f1-4906-bb7a-2b81845cb38d admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:00.364574 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-77b751c9-55bd-4699-b275-74f81593d1b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a53f72887f114989b8c013b87ac0be04'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:00.365428 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-77b751c9-55bd-4699-b275-74f81593d1b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a53f72887f114989b8c013b87ac0be04'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:00.366204 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-77b751c9-55bd-4699-b275-74f81593d1b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a53f72887f114989b8c013b87ac0be04'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:00.373951 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-77b751c9-55bd-4699-b275-74f81593d1b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a53f72887f114989b8c013b87ac0be04'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:00.389499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-77b751c9-55bd-4699-b275-74f81593d1b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a53f72887f114989b8c013b87ac0be04'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:00.482898 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1102/2220] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:07:00 2022] POST /identity/v3/users => generated 285 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:07:00.486990 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99adadf6-35f1-4906-bb7a-2b81845cb38d admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:00.488983 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-99adadf6-35f1-4906-bb7a-2b81845cb38d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:00.502600 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99adadf6-35f1-4906-bb7a-2b81845cb38d admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:00.508418 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71326685-8eac-4185-abc0-6443484c266b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:00.509634 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71326685-8eac-4185-abc0-6443484c266b admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:00.518981 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71326685-8eac-4185-abc0-6443484c266b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:00.519528 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1119/2221] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:07:00 2022] DELETE /identity/v3/users/a53f72887f114989b8c013b87ac0be04 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:07:00.519638 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71326685-8eac-4185-abc0-6443484c266b admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:00.519638 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71326685-8eac-4185-abc0-6443484c266b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:00.519638 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71326685-8eac-4185-abc0-6443484c266b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:00.554535 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1103/2222] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:07:00.558098 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b751c9-55bd-4699-b275-74f81593d1b9 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:00.638775 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b751c9-55bd-4699-b275-74f81593d1b9 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:00.643964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e33d50-7a8b-463f-89be-21b03542579d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:00.644701 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e33d50-7a8b-463f-89be-21b03542579d admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:00.644978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e33d50-7a8b-463f-89be-21b03542579d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:00.645248 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e33d50-7a8b-463f-89be-21b03542579d admin admin] PATH_INFO: `/v3/projects/719ee356a279427f94775e069471481c/users/6111b47e53a34f2997f5b7ed7d4514bf/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:00.660542 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e33d50-7a8b-463f-89be-21b03542579d admin admin] RBAC: Authorizing `identity:create_grant(project_id=719ee356a279427f94775e069471481c, user_id=6111b47e53a34f2997f5b7ed7d4514bf, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:00.663256 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e33d50-7a8b-463f-89be-21b03542579d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:00.761820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1120/2223] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:07:00 2022] PUT /identity/v3/projects/719ee356a279427f94775e069471481c/users/6111b47e53a34f2997f5b7ed7d4514bf/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:07:00.765802 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71326685-8eac-4185-abc0-6443484c266b admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:00.775765 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71326685-8eac-4185-abc0-6443484c266b admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:00.834935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1290161-757e-4f6e-9ed7-fe185338d5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:00.835547 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1290161-757e-4f6e-9ed7-fe185338d5cf admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:00.835830 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1290161-757e-4f6e-9ed7-fe185338d5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:00.836130 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1290161-757e-4f6e-9ed7-fe185338d5cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:00.836779 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1290161-757e-4f6e-9ed7-fe185338d5cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:00.838584 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1290161-757e-4f6e-9ed7-fe185338d5cf admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:00.852208 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1104/2224] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:07:00.853366 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e33d50-7a8b-463f-89be-21b03542579d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:00.865989 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e33d50-7a8b-463f-89be-21b03542579d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:00.870149 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1290161-757e-4f6e-9ed7-fe185338d5cf admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:00.872045 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d1290161-757e-4f6e-9ed7-fe185338d5cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:00.872213 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7b6ff-6a30-46a8-9582-f749a10a3fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:00.872744 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7b6ff-6a30-46a8-9582-f749a10a3fbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:00.873047 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7b6ff-6a30-46a8-9582-f749a10a3fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:00.873390 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7b6ff-6a30-46a8-9582-f749a10a3fbf admin admin] PATH_INFO: `/v3/projects/719ee356a279427f94775e069471481c/users/6111b47e53a34f2997f5b7ed7d4514bf/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:00.875415 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f7b6ff-6a30-46a8-9582-f749a10a3fbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=719ee356a279427f94775e069471481c, user_id=6111b47e53a34f2997f5b7ed7d4514bf, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:00.878271 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f7b6ff-6a30-46a8-9582-f749a10a3fbf admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:00.885962 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1290161-757e-4f6e-9ed7-fe185338d5cf admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:00.900709 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1121/2225] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:07:00 2022] PUT /identity/v3/projects/719ee356a279427f94775e069471481c/users/6111b47e53a34f2997f5b7ed7d4514bf/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:07:00.911951 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711426a5-4ba6-483e-8c69-6a3f992a5174 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:00.912303 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711426a5-4ba6-483e-8c69-6a3f992a5174 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:00.912542 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711426a5-4ba6-483e-8c69-6a3f992a5174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:00.949390 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-711426a5-4ba6-483e-8c69-6a3f992a5174 None None] MFA Rules not processed for user `6111b47e53a34f2997f5b7ed7d4514bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:00.950663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-711426a5-4ba6-483e-8c69-6a3f992a5174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:00.969655 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:00.972720 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72435a7-28b4-4d03-8a9a-fe40fb455764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:00.973392 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72435a7-28b4-4d03-8a9a-fe40fb455764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:00.974203 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72435a7-28b4-4d03-8a9a-fe40fb455764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:00.974478 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72435a7-28b4-4d03-8a9a-fe40fb455764 admin admin] PATH_INFO: `/v3/users/b75c1f61554d489cb035e6af6819bd02` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:00.975624 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72435a7-28b4-4d03-8a9a-fe40fb455764 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b75c1f61554d489cb035e6af6819bd02)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:00.977689 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72435a7-28b4-4d03-8a9a-fe40fb455764 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:01.022488 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1122/2226] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:00 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:07:01.034147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711426a5-4ba6-483e-8c69-6a3f992a5174 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:01.035467 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-711426a5-4ba6-483e-8c69-6a3f992a5174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:01.092745 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711426a5-4ba6-483e-8c69-6a3f992a5174 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:01.096409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77f282-079b-495b-a3f4-a8d58304409f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:01.097008 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77f282-079b-495b-a3f4-a8d58304409f service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:01.097286 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77f282-079b-495b-a3f4-a8d58304409f service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:01.097557 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77f282-079b-495b-a3f4-a8d58304409f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:01.107044 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d77f282-079b-495b-a3f4-a8d58304409f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:01.108536 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d77f282-079b-495b-a3f4-a8d58304409f service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:01.118447 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1123/2227] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:07:01 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:07:01.119281 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77f282-079b-495b-a3f4-a8d58304409f service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:01.120570 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8d77f282-079b-495b-a3f4-a8d58304409f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:01.138605 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77f282-079b-495b-a3f4-a8d58304409f service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:01.143220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af2857-d716-4b0c-8bef-3a188005bd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:01.143835 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af2857-d716-4b0c-8bef-3a188005bd28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:01.144140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af2857-d716-4b0c-8bef-3a188005bd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:01.144415 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af2857-d716-4b0c-8bef-3a188005bd28 admin admin] PATH_INFO: `/v3/projects/170bd8b796af47fbacee0b05b65dcab0` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:01.145554 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6af2857-d716-4b0c-8bef-3a188005bd28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=170bd8b796af47fbacee0b05b65dcab0)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:01.147454 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6af2857-d716-4b0c-8bef-3a188005bd28 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:01.180502 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b72435a7-28b4-4d03-8a9a-fe40fb455764 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b75c1f61554d489cb035e6af6819bd02'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:01.181172 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b72435a7-28b4-4d03-8a9a-fe40fb455764 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b75c1f61554d489cb035e6af6819bd02'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:01.188163 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b72435a7-28b4-4d03-8a9a-fe40fb455764 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b75c1f61554d489cb035e6af6819bd02'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:01.199795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b72435a7-28b4-4d03-8a9a-fe40fb455764 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b75c1f61554d489cb035e6af6819bd02'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:01.200468 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b72435a7-28b4-4d03-8a9a-fe40fb455764 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b75c1f61554d489cb035e6af6819bd02'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:01.269010 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c6af2857-d716-4b0c-8bef-3a188005bd28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '170bd8b796af47fbacee0b05b65dcab0', 'request_id': 'req-c6af2857-d716-4b0c-8bef-3a188005bd28'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:01.275225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c6af2857-d716-4b0c-8bef-3a188005bd28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '170bd8b796af47fbacee0b05b65dcab0', 'request_id': 'req-c6af2857-d716-4b0c-8bef-3a188005bd28'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:01.292667 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1105/2228] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:07:00 2022] DELETE /identity/v3/users/b75c1f61554d489cb035e6af6819bd02 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:07:01.333523 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72435a7-28b4-4d03-8a9a-fe40fb455764 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:01.335072 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b72435a7-28b4-4d03-8a9a-fe40fb455764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:01.370459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c6af2857-d716-4b0c-8bef-3a188005bd28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '170bd8b796af47fbacee0b05b65dcab0', 'request_id': 'req-c6af2857-d716-4b0c-8bef-3a188005bd28'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:01.384297 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c6af2857-d716-4b0c-8bef-3a188005bd28 admin admin] The token cache is being invalidate because project 170bd8b796af47fbacee0b05b65dcab0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:07:01.384598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c6af2857-d716-4b0c-8bef-3a188005bd28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:01.385309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c6af2857-d716-4b0c-8bef-3a188005bd28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:01.386872 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1124/2229] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:07:01 2022] DELETE /identity/v3/projects/170bd8b796af47fbacee0b05b65dcab0 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:07:01.392708 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5f5eb-9593-4bef-9603-ec4e29328891 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:01.393088 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5f5eb-9593-4bef-9603-ec4e29328891 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:01.393375 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5f5eb-9593-4bef-9603-ec4e29328891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:01.405771 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72435a7-28b4-4d03-8a9a-fe40fb455764 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:01.409495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df53fb0-54f2-42f5-9fd7-45642af11bbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:01.410107 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df53fb0-54f2-42f5-9fd7-45642af11bbd service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:01.410806 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df53fb0-54f2-42f5-9fd7-45642af11bbd service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:01.411695 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df53fb0-54f2-42f5-9fd7-45642af11bbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:01.413806 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3df53fb0-54f2-42f5-9fd7-45642af11bbd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:01.417641 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c9c5f5eb-9593-4bef-9603-ec4e29328891 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:01.418822 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c9c5f5eb-9593-4bef-9603-ec4e29328891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:01.480511 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df53fb0-54f2-42f5-9fd7-45642af11bbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:01.481951 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df53fb0-54f2-42f5-9fd7-45642af11bbd service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:01.491566 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1106/2230] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:07:01 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:07:01.498851 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1125/2231] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:07:01.506876 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696069a-6633-40d7-936f-ec73c58f1f50 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:01.507210 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696069a-6633-40d7-936f-ec73c58f1f50 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:01.507731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696069a-6633-40d7-936f-ec73c58f1f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:01.528632 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-1696069a-6633-40d7-936f-ec73c58f1f50 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:01.529843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-1696069a-6633-40d7-936f-ec73c58f1f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:01.538912 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1107/2232] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:07:01.543914 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c5f5eb-9593-4bef-9603-ec4e29328891 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:01.554499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c5f5eb-9593-4bef-9603-ec4e29328891 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:01.559473 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ac9cf-a7da-4349-bf7b-a2ebbafeea2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:01.560258 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ac9cf-a7da-4349-bf7b-a2ebbafeea2c admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:01.560572 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ac9cf-a7da-4349-bf7b-a2ebbafeea2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:01.560846 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ac9cf-a7da-4349-bf7b-a2ebbafeea2c admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:01.561517 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563ac9cf-a7da-4349-bf7b-a2ebbafeea2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:07:01.561830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563ac9cf-a7da-4349-bf7b-a2ebbafeea2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:01.563457 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563ac9cf-a7da-4349-bf7b-a2ebbafeea2c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:01.569569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:01.577978 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696069a-6633-40d7-936f-ec73c58f1f50 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:01.581187 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1126/2233] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:07:01.587044 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ac9cf-a7da-4349-bf7b-a2ebbafeea2c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:01.588761 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-1696069a-6633-40d7-936f-ec73c58f1f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:01.606090 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696069a-6633-40d7-936f-ec73c58f1f50 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:01.607856 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ac9cf-a7da-4349-bf7b-a2ebbafeea2c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:01.611032 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea2e3c-e3c7-4981-8a2a-c0862665c31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:01.611724 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea2e3c-e3c7-4981-8a2a-c0862665c31e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:01.612009 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea2e3c-e3c7-4981-8a2a-c0862665c31e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:01.612476 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea2e3c-e3c7-4981-8a2a-c0862665c31e admin admin] PATH_INFO: `/v3/projects/47af18d14ddd400898b8cf3256af985b` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:01.613830 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baea2e3c-e3c7-4981-8a2a-c0862665c31e admin admin] RBAC: Authorizing `identity:delete_project(project_id=47af18d14ddd400898b8cf3256af985b)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:01.618552 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c792b9-82aa-4c5f-8e7a-10b8285c111d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:01.619249 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c792b9-82aa-4c5f-8e7a-10b8285c111d admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:01.619599 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c792b9-82aa-4c5f-8e7a-10b8285c111d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:01.619708 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baea2e3c-e3c7-4981-8a2a-c0862665c31e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:01.620029 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c792b9-82aa-4c5f-8e7a-10b8285c111d admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:01.623950 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c792b9-82aa-4c5f-8e7a-10b8285c111d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:01.623950 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c792b9-82aa-4c5f-8e7a-10b8285c111d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:01.729516 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:01.736586 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1127/2234] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:07:01 2022] POST /identity/v3/projects => generated 375 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:07:01.741079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c792b9-82aa-4c5f-8e7a-10b8285c111d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:01.760680 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-baea2e3c-e3c7-4981-8a2a-c0862665c31e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47af18d14ddd400898b8cf3256af985b', 'request_id': 'req-baea2e3c-e3c7-4981-8a2a-c0862665c31e'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:01.813419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:01.819990 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c792b9-82aa-4c5f-8e7a-10b8285c111d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:01.820448 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-baea2e3c-e3c7-4981-8a2a-c0862665c31e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47af18d14ddd400898b8cf3256af985b', 'request_id': 'req-baea2e3c-e3c7-4981-8a2a-c0862665c31e'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:01.826753 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-baea2e3c-e3c7-4981-8a2a-c0862665c31e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47af18d14ddd400898b8cf3256af985b', 'request_id': 'req-baea2e3c-e3c7-4981-8a2a-c0862665c31e'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:01.840414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-baea2e3c-e3c7-4981-8a2a-c0862665c31e admin admin] The token cache is being invalidate because project 47af18d14ddd400898b8cf3256af985b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:07:01.842029 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-baea2e3c-e3c7-4981-8a2a-c0862665c31e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:01.842029 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-baea2e3c-e3c7-4981-8a2a-c0862665c31e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:01.843165 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1108/2235] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:07:01 2022] DELETE /identity/v3/projects/47af18d14ddd400898b8cf3256af985b => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:07:01.846721 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea2e3c-e3c7-4981-8a2a-c0862665c31e admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:01.848257 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-baea2e3c-e3c7-4981-8a2a-c0862665c31e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:01.887096 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc31da9-13b3-4b13-bd3c-3a263fee1666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:01.887996 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc31da9-13b3-4b13-bd3c-3a263fee1666 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:01.888398 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc31da9-13b3-4b13-bd3c-3a263fee1666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:01.888813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc31da9-13b3-4b13-bd3c-3a263fee1666 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:01.889576 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc31da9-13b3-4b13-bd3c-3a263fee1666 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:01.891859 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc31da9-13b3-4b13-bd3c-3a263fee1666 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:01.916321 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea2e3c-e3c7-4981-8a2a-c0862665c31e admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:01.921017 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d32a758-a0e5-4283-b0a5-8244113f424c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:01.921644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32a758-a0e5-4283-b0a5-8244113f424c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:01.921921 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32a758-a0e5-4283-b0a5-8244113f424c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:01.922192 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32a758-a0e5-4283-b0a5-8244113f424c admin admin] PATH_INFO: `/v3/users/f950ed50da104a469cb3efa7bc3ab6f9` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:01.923343 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d32a758-a0e5-4283-b0a5-8244113f424c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f950ed50da104a469cb3efa7bc3ab6f9)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:01.925316 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d32a758-a0e5-4283-b0a5-8244113f424c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:01.985477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1128/2236] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:07:01 2022] POST /identity/v3/users => generated 288 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:07:01.987984 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc31da9-13b3-4b13-bd3c-3a263fee1666 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:01.990322 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2bc31da9-13b3-4b13-bd3c-3a263fee1666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:02.002152 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc31da9-13b3-4b13-bd3c-3a263fee1666 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:02.006927 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891c1a5-e76e-4ad7-9028-46b5800987a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:02.007514 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891c1a5-e76e-4ad7-9028-46b5800987a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:02.007795 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891c1a5-e76e-4ad7-9028-46b5800987a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:02.008102 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891c1a5-e76e-4ad7-9028-46b5800987a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:02.008746 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b891c1a5-e76e-4ad7-9028-46b5800987a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:02.010535 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b891c1a5-e76e-4ad7-9028-46b5800987a9 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:02.023104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1129/2237] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:07:02.026898 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891c1a5-e76e-4ad7-9028-46b5800987a9 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:02.036877 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891c1a5-e76e-4ad7-9028-46b5800987a9 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:02.041501 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34dc48-0a3a-4897-b5d4-1c68cb798aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:02.042058 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34dc48-0a3a-4897-b5d4-1c68cb798aac admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:02.042338 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34dc48-0a3a-4897-b5d4-1c68cb798aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:02.042612 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34dc48-0a3a-4897-b5d4-1c68cb798aac admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:02.043233 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da34dc48-0a3a-4897-b5d4-1c68cb798aac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:02.044879 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da34dc48-0a3a-4897-b5d4-1c68cb798aac admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:02.055572 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1130/2238] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:07:02.059424 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34dc48-0a3a-4897-b5d4-1c68cb798aac admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:02.068927 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34dc48-0a3a-4897-b5d4-1c68cb798aac admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:02.073968 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15587445-11dc-4b0c-a6e4-e3ba9138f909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:02.074598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15587445-11dc-4b0c-a6e4-e3ba9138f909 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:02.074877 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15587445-11dc-4b0c-a6e4-e3ba9138f909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:02.075149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15587445-11dc-4b0c-a6e4-e3ba9138f909 admin admin] PATH_INFO: `/v3/projects/eececfc64fc34822815d599124b70490/users/efa09c2f895145e7b6f2978d0cc3a3d8/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:02.077361 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1d32a758-a0e5-4283-b0a5-8244113f424c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f950ed50da104a469cb3efa7bc3ab6f9'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:02.078036 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1d32a758-a0e5-4283-b0a5-8244113f424c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f950ed50da104a469cb3efa7bc3ab6f9'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:02.085716 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1d32a758-a0e5-4283-b0a5-8244113f424c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f950ed50da104a469cb3efa7bc3ab6f9'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:02.090696 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15587445-11dc-4b0c-a6e4-e3ba9138f909 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eececfc64fc34822815d599124b70490, user_id=efa09c2f895145e7b6f2978d0cc3a3d8, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:02.093735 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15587445-11dc-4b0c-a6e4-e3ba9138f909 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:02.097605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1d32a758-a0e5-4283-b0a5-8244113f424c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f950ed50da104a469cb3efa7bc3ab6f9'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:02.098350 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1d32a758-a0e5-4283-b0a5-8244113f424c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f950ed50da104a469cb3efa7bc3ab6f9'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:02.109459 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:02.176097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1131/2239] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:07:02 2022] PUT /identity/v3/projects/eececfc64fc34822815d599124b70490/users/efa09c2f895145e7b6f2978d0cc3a3d8/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:07:02.186434 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51669a29-d86a-42cc-bec0-9640627f0f54 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:02.186728 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51669a29-d86a-42cc-bec0-9640627f0f54 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:02.187008 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51669a29-d86a-42cc-bec0-9640627f0f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:02.187542 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1109/2240] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:07:01 2022] DELETE /identity/v3/users/f950ed50da104a469cb3efa7bc3ab6f9 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:07:02.219229 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-51669a29-d86a-42cc-bec0-9640627f0f54 None None] MFA Rules not processed for user `efa09c2f895145e7b6f2978d0cc3a3d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:02.219229 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-51669a29-d86a-42cc-bec0-9640627f0f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:02.284777 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1132/2241] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:07:02.296557 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d32a758-a0e5-4283-b0a5-8244113f424c admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:02.297953 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-1d32a758-a0e5-4283-b0a5-8244113f424c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:02.308350 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:02.358487 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d32a758-a0e5-4283-b0a5-8244113f424c admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:02.362147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cb2afe-4c33-468d-a887-97a407e008ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:02.362752 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb2afe-4c33-468d-a887-97a407e008ea service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:02.363032 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb2afe-4c33-468d-a887-97a407e008ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:02.363306 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb2afe-4c33-468d-a887-97a407e008ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:02.371472 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cb2afe-4c33-468d-a887-97a407e008ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:02.372941 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cb2afe-4c33-468d-a887-97a407e008ea service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:02.381839 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1110/2242] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:07:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:07:02.435344 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:02.476829 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51669a29-d86a-42cc-bec0-9640627f0f54 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:02.478768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-51669a29-d86a-42cc-bec0-9640627f0f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:02.553789 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:02.559388 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51669a29-d86a-42cc-bec0-9640627f0f54 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:02.564042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32775ca8-ad83-4330-b5e5-265408982a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:02.565958 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32775ca8-ad83-4330-b5e5-265408982a24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:02.566266 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32775ca8-ad83-4330-b5e5-265408982a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:02.566551 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32775ca8-ad83-4330-b5e5-265408982a24 admin admin] PATH_INFO: `/v3/projects/6e48e202b622484ba250047d3a33c091` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:02.568266 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32775ca8-ad83-4330-b5e5-265408982a24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e48e202b622484ba250047d3a33c091)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:02.570209 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32775ca8-ad83-4330-b5e5-265408982a24 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:02.657897 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-32775ca8-ad83-4330-b5e5-265408982a24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e48e202b622484ba250047d3a33c091', 'request_id': 'req-32775ca8-ad83-4330-b5e5-265408982a24'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:02.664307 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-32775ca8-ad83-4330-b5e5-265408982a24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e48e202b622484ba250047d3a33c091', 'request_id': 'req-32775ca8-ad83-4330-b5e5-265408982a24'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:02.732249 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:02.749183 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-32775ca8-ad83-4330-b5e5-265408982a24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e48e202b622484ba250047d3a33c091', 'request_id': 'req-32775ca8-ad83-4330-b5e5-265408982a24'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:02.764181 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-32775ca8-ad83-4330-b5e5-265408982a24 admin admin] The token cache is being invalidate because project 6e48e202b622484ba250047d3a33c091 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:07:02.764570 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-32775ca8-ad83-4330-b5e5-265408982a24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:02.766766 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-32775ca8-ad83-4330-b5e5-265408982a24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:02.768724 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1133/2243] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:07:02 2022] DELETE /identity/v3/projects/6e48e202b622484ba250047d3a33c091 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:07:02.774393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d349e-9f23-42f9-b848-3d7db2ed2c4e None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:02.774716 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d349e-9f23-42f9-b848-3d7db2ed2c4e None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:02.774983 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d349e-9f23-42f9-b848-3d7db2ed2c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:02.796419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-491d349e-9f23-42f9-b848-3d7db2ed2c4e None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:02.797651 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-491d349e-9f23-42f9-b848-3d7db2ed2c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:02.862495 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:02.883725 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1111/2244] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:07:02.891769 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc934e96-da1c-4ce6-8d07-b22fe6ed7e7a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:02.892055 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc934e96-da1c-4ce6-8d07-b22fe6ed7e7a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:02.892354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc934e96-da1c-4ce6-8d07-b22fe6ed7e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:02.914732 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-fc934e96-da1c-4ce6-8d07-b22fe6ed7e7a None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:02.915922 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fc934e96-da1c-4ce6-8d07-b22fe6ed7e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:02.925359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1134/2245] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:07:02.928265 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:02.937712 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d349e-9f23-42f9-b848-3d7db2ed2c4e None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:02.948923 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d349e-9f23-42f9-b848-3d7db2ed2c4e None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:02.954032 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b783b1e-08dd-4a55-a425-429b39dd4c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:02.954641 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b783b1e-08dd-4a55-a425-429b39dd4c72 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:02.954918 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b783b1e-08dd-4a55-a425-429b39dd4c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:02.955186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b783b1e-08dd-4a55-a425-429b39dd4c72 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:02.955836 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b783b1e-08dd-4a55-a425-429b39dd4c72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:07:02.956213 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b783b1e-08dd-4a55-a425-429b39dd4c72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:02.957886 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b783b1e-08dd-4a55-a425-429b39dd4c72 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:02.969960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1112/2246] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:07:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:07:02.974374 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc934e96-da1c-4ce6-8d07-b22fe6ed7e7a None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:02.990348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc934e96-da1c-4ce6-8d07-b22fe6ed7e7a None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:02.995936 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ccd6c-7481-49bc-b1e8-9afbc51e832e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:02.996580 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ccd6c-7481-49bc-b1e8-9afbc51e832e admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:02.996859 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ccd6c-7481-49bc-b1e8-9afbc51e832e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:02.997130 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ccd6c-7481-49bc-b1e8-9afbc51e832e admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:02.997765 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ccd6c-7481-49bc-b1e8-9afbc51e832e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:02.999678 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ccd6c-7481-49bc-b1e8-9afbc51e832e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:03.105797 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1135/2247] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:07:02 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:07:03.109909 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b783b1e-08dd-4a55-a425-429b39dd4c72 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:03.120697 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b783b1e-08dd-4a55-a425-429b39dd4c72 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:03.178393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc42ac88-f79b-4661-b849-3b4b1f27a720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:03.180805 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc42ac88-f79b-4661-b849-3b4b1f27a720 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:03.180805 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc42ac88-f79b-4661-b849-3b4b1f27a720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:03.180805 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc42ac88-f79b-4661-b849-3b4b1f27a720 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:03.180805 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc42ac88-f79b-4661-b849-3b4b1f27a720 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:03.182181 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc42ac88-f79b-4661-b849-3b4b1f27a720 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:03.288881 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1113/2248] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:07:03 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:07:03.293099 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ccd6c-7481-49bc-b1e8-9afbc51e832e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:03.304745 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ccd6c-7481-49bc-b1e8-9afbc51e832e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:03.309535 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c68518-9d62-4fe8-82dd-13dc3eb0ad40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:03.310283 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c68518-9d62-4fe8-82dd-13dc3eb0ad40 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:03.310558 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c68518-9d62-4fe8-82dd-13dc3eb0ad40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:03.310827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c68518-9d62-4fe8-82dd-13dc3eb0ad40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:03.311475 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c68518-9d62-4fe8-82dd-13dc3eb0ad40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:03.313112 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c68518-9d62-4fe8-82dd-13dc3eb0ad40 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:03.323184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1136/2249] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:07:03.327419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc42ac88-f79b-4661-b849-3b4b1f27a720 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:03.338282 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc42ac88-f79b-4661-b849-3b4b1f27a720 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:03.343048 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941613c5-685b-41cc-8cf4-e3e63d932c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:03.343634 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941613c5-685b-41cc-8cf4-e3e63d932c37 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:03.343912 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941613c5-685b-41cc-8cf4-e3e63d932c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:03.344614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941613c5-685b-41cc-8cf4-e3e63d932c37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:03.345709 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941613c5-685b-41cc-8cf4-e3e63d932c37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:03.347287 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941613c5-685b-41cc-8cf4-e3e63d932c37 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:03.359525 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1114/2250] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:07:03.363538 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c68518-9d62-4fe8-82dd-13dc3eb0ad40 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:03.374520 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c68518-9d62-4fe8-82dd-13dc3eb0ad40 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:03.379344 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb47042-6054-4e8a-8e68-4b04f2c3fef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:03.386859 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb47042-6054-4e8a-8e68-4b04f2c3fef6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:03.386859 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb47042-6054-4e8a-8e68-4b04f2c3fef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:03.386859 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb47042-6054-4e8a-8e68-4b04f2c3fef6 admin admin] PATH_INFO: `/v3/projects/963f384bc92e4c02ba1248cff606f9de/users/e69da7f47f274dccb7c2f104c7386c61/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:03.394665 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb47042-6054-4e8a-8e68-4b04f2c3fef6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=963f384bc92e4c02ba1248cff606f9de, user_id=e69da7f47f274dccb7c2f104c7386c61, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:03.397367 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb47042-6054-4e8a-8e68-4b04f2c3fef6 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:03.457360 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1137/2251] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:07:03 2022] PUT /identity/v3/projects/963f384bc92e4c02ba1248cff606f9de/users/e69da7f47f274dccb7c2f104c7386c61/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:07:03.459131 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:03.470732 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a5fba-a1c7-4686-b35b-7f75fcc51b2b None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:03.471020 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a5fba-a1c7-4686-b35b-7f75fcc51b2b None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:03.471287 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a5fba-a1c7-4686-b35b-7f75fcc51b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:03.499557 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-b60a5fba-a1c7-4686-b35b-7f75fcc51b2b None None] MFA Rules not processed for user `e69da7f47f274dccb7c2f104c7386c61`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:03.500820 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b60a5fba-a1c7-4686-b35b-7f75fcc51b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:03.563285 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1115/2252] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:07:03.574378 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb47042-6054-4e8a-8e68-4b04f2c3fef6 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:03.576257 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4bb47042-6054-4e8a-8e68-4b04f2c3fef6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:03.654104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb47042-6054-4e8a-8e68-4b04f2c3fef6 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:03.661428 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8db84a-0576-47c1-9693-536a4feb52bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:03.662033 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8db84a-0576-47c1-9693-536a4feb52bf service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:03.662311 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8db84a-0576-47c1-9693-536a4feb52bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:03.662583 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8db84a-0576-47c1-9693-536a4feb52bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:03.672401 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8db84a-0576-47c1-9693-536a4feb52bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:03.673833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8db84a-0576-47c1-9693-536a4feb52bf service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:03.679618 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:03.682735 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1138/2253] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:07:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:07:03.942906 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:04.161229 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:04.463768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:04.749509 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:04.954949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:04.983632 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:05.160712 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:05.212286 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:05.376284 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:05.400513 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:05.742068 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:05.783732 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:06.039581 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:16.101716 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=35 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:16.480206 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=6 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:18.788590 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a5fba-a1c7-4686-b35b-7f75fcc51b2b None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:18.864711 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a5fba-a1c7-4686-b35b-7f75fcc51b2b None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:18.879645 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f457e-ca66-473e-9b62-e5fe09c08d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:18.881669 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f457e-ca66-473e-9b62-e5fe09c08d04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:18.881669 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f457e-ca66-473e-9b62-e5fe09c08d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:18.881669 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f457e-ca66-473e-9b62-e5fe09c08d04 admin admin] PATH_INFO: `/v3/users/e69da7f47f274dccb7c2f104c7386c61` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:18.883006 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f457e-ca66-473e-9b62-e5fe09c08d04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e69da7f47f274dccb7c2f104c7386c61)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:18.885698 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f457e-ca66-473e-9b62-e5fe09c08d04 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:19.089237 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-444f457e-ca66-473e-9b62-e5fe09c08d04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e69da7f47f274dccb7c2f104c7386c61'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:19.090162 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-444f457e-ca66-473e-9b62-e5fe09c08d04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e69da7f47f274dccb7c2f104c7386c61'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:19.098434 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-444f457e-ca66-473e-9b62-e5fe09c08d04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e69da7f47f274dccb7c2f104c7386c61'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:19.113747 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-444f457e-ca66-473e-9b62-e5fe09c08d04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e69da7f47f274dccb7c2f104c7386c61'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:19.114419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-444f457e-ca66-473e-9b62-e5fe09c08d04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e69da7f47f274dccb7c2f104c7386c61'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:19.212084 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1116/2254] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:07:18 2022] DELETE /identity/v3/users/e69da7f47f274dccb7c2f104c7386c61 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:07:19.216359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506df91a-0199-4c90-9370-4bdd34d7093f None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:19.217038 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506df91a-0199-4c90-9370-4bdd34d7093f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:19.217387 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506df91a-0199-4c90-9370-4bdd34d7093f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:19.236467 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-506df91a-0199-4c90-9370-4bdd34d7093f None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:19.237661 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-506df91a-0199-4c90-9370-4bdd34d7093f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:19.301882 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1139/2255] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:07:19.312908 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f457e-ca66-473e-9b62-e5fe09c08d04 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:19.314731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-444f457e-ca66-473e-9b62-e5fe09c08d04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:19.376927 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f457e-ca66-473e-9b62-e5fe09c08d04 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:19.381913 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934042a-9f85-45a4-b59b-3ae6722f9dda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:19.382819 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934042a-9f85-45a4-b59b-3ae6722f9dda service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:19.383106 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934042a-9f85-45a4-b59b-3ae6722f9dda service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:19.383383 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934042a-9f85-45a4-b59b-3ae6722f9dda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:19.393533 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8934042a-9f85-45a4-b59b-3ae6722f9dda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:19.395001 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8934042a-9f85-45a4-b59b-3ae6722f9dda service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:19.405561 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1117/2256] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:07:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:07:19.764250 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506df91a-0199-4c90-9370-4bdd34d7093f None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:19.764250 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-506df91a-0199-4c90-9370-4bdd34d7093f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:19.785694 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506df91a-0199-4c90-9370-4bdd34d7093f None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:19.794006 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2643ce90-0a0c-47c2-9306-c08fbde97c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:19.794006 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643ce90-0a0c-47c2-9306-c08fbde97c11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:19.795148 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643ce90-0a0c-47c2-9306-c08fbde97c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:19.795444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643ce90-0a0c-47c2-9306-c08fbde97c11 admin admin] PATH_INFO: `/v3/projects/963f384bc92e4c02ba1248cff606f9de` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:19.797161 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2643ce90-0a0c-47c2-9306-c08fbde97c11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=963f384bc92e4c02ba1248cff606f9de)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:19.799513 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2643ce90-0a0c-47c2-9306-c08fbde97c11 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:19.963202 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2643ce90-0a0c-47c2-9306-c08fbde97c11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '963f384bc92e4c02ba1248cff606f9de', 'request_id': 'req-2643ce90-0a0c-47c2-9306-c08fbde97c11'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:19.970193 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2643ce90-0a0c-47c2-9306-c08fbde97c11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '963f384bc92e4c02ba1248cff606f9de', 'request_id': 'req-2643ce90-0a0c-47c2-9306-c08fbde97c11'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:20.039506 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2643ce90-0a0c-47c2-9306-c08fbde97c11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '963f384bc92e4c02ba1248cff606f9de', 'request_id': 'req-2643ce90-0a0c-47c2-9306-c08fbde97c11'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:20.050619 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2643ce90-0a0c-47c2-9306-c08fbde97c11 admin admin] The token cache is being invalidate because project 963f384bc92e4c02ba1248cff606f9de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:07:20.050905 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2643ce90-0a0c-47c2-9306-c08fbde97c11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:20.051518 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2643ce90-0a0c-47c2-9306-c08fbde97c11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:20.053061 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1140/2257] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:07:19 2022] DELETE /identity/v3/projects/963f384bc92e4c02ba1248cff606f9de => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:07:20.060366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b6c1d5-0107-446a-b5be-783533e03680 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:20.060866 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b6c1d5-0107-446a-b5be-783533e03680 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:20.061062 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b6c1d5-0107-446a-b5be-783533e03680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:20.081005 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-11b6c1d5-0107-446a-b5be-783533e03680 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:20.082214 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-11b6c1d5-0107-446a-b5be-783533e03680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:20.100959 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:20.144486 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1118/2258] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:07:20.152704 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d0b4c-504b-4c9f-8c67-79afe1c1691e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:20.152988 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d0b4c-504b-4c9f-8c67-79afe1c1691e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:20.153291 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d0b4c-504b-4c9f-8c67-79afe1c1691e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:20.175135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-8f7d0b4c-504b-4c9f-8c67-79afe1c1691e None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:20.176477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8f7d0b4c-504b-4c9f-8c67-79afe1c1691e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:20.184497 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1141/2259] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:07:20.188880 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b6c1d5-0107-446a-b5be-783533e03680 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:20.200368 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b6c1d5-0107-446a-b5be-783533e03680 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:20.205364 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd3798-8981-4494-89b9-da3554d6161d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:20.205973 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd3798-8981-4494-89b9-da3554d6161d admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:20.206252 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd3798-8981-4494-89b9-da3554d6161d admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:20.206522 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd3798-8981-4494-89b9-da3554d6161d admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:20.207168 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bd3798-8981-4494-89b9-da3554d6161d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:07:20.207471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bd3798-8981-4494-89b9-da3554d6161d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:20.209706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bd3798-8981-4494-89b9-da3554d6161d admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:20.226598 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1119/2260] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:07:20.226897 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d0b4c-504b-4c9f-8c67-79afe1c1691e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:20.238080 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d0b4c-504b-4c9f-8c67-79afe1c1691e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:20.243965 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336ea22-e36b-4118-865b-a0d30df8e5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:20.244600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336ea22-e36b-4118-865b-a0d30df8e5e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:20.244889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336ea22-e36b-4118-865b-a0d30df8e5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:20.245152 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336ea22-e36b-4118-865b-a0d30df8e5e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:20.245785 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7336ea22-e36b-4118-865b-a0d30df8e5e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:20.247692 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7336ea22-e36b-4118-865b-a0d30df8e5e7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:20.351663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1142/2261] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:07:20 2022] POST /identity/v3/projects => generated 383 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:07:20.356539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd3798-8981-4494-89b9-da3554d6161d admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:20.374886 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd3798-8981-4494-89b9-da3554d6161d admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:20.433611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62008df-e12e-4e79-b600-60c7471e6b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:20.434232 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62008df-e12e-4e79-b600-60c7471e6b84 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:20.434533 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62008df-e12e-4e79-b600-60c7471e6b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:20.434789 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62008df-e12e-4e79-b600-60c7471e6b84 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:20.435426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62008df-e12e-4e79-b600-60c7471e6b84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:20.437433 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62008df-e12e-4e79-b600-60c7471e6b84 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:20.543990 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1120/2262] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:07:20 2022] POST /identity/v3/users => generated 292 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:07:20.547952 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336ea22-e36b-4118-865b-a0d30df8e5e7 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:20.558148 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336ea22-e36b-4118-865b-a0d30df8e5e7 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:20.562901 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba5d605-453f-4fce-a992-b442ccd1686d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:20.563511 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5d605-453f-4fce-a992-b442ccd1686d admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:20.563792 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5d605-453f-4fce-a992-b442ccd1686d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:20.564063 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5d605-453f-4fce-a992-b442ccd1686d admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:20.564755 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba5d605-453f-4fce-a992-b442ccd1686d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:20.570051 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba5d605-453f-4fce-a992-b442ccd1686d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:20.585157 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1143/2263] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:07:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:07:20.586594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62008df-e12e-4e79-b600-60c7471e6b84 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:20.597668 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62008df-e12e-4e79-b600-60c7471e6b84 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:20.604823 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc24109-6e95-4614-97cf-fb144af61b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:20.605397 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc24109-6e95-4614-97cf-fb144af61b8f admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:20.605708 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc24109-6e95-4614-97cf-fb144af61b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:20.606068 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc24109-6e95-4614-97cf-fb144af61b8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:20.606792 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc24109-6e95-4614-97cf-fb144af61b8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:20.608647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc24109-6e95-4614-97cf-fb144af61b8f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:20.625254 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1121/2264] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:07:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:07:20.626677 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba5d605-453f-4fce-a992-b442ccd1686d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:20.643410 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba5d605-453f-4fce-a992-b442ccd1686d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:20.663819 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a8d47-74e9-41d6-8833-ac67b7272c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:20.664728 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a8d47-74e9-41d6-8833-ac67b7272c43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:20.665077 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a8d47-74e9-41d6-8833-ac67b7272c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:20.665423 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a8d47-74e9-41d6-8833-ac67b7272c43 admin admin] PATH_INFO: `/v3/projects/25e832f49fbc4fee98bbdf0334bd68af/users/f05b30c787cc4de68fd725600842d0ca/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:20.687129 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a8d47-74e9-41d6-8833-ac67b7272c43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25e832f49fbc4fee98bbdf0334bd68af, user_id=f05b30c787cc4de68fd725600842d0ca, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:20.690298 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a8d47-74e9-41d6-8833-ac67b7272c43 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:20.761506 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1144/2265] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:07:20 2022] PUT /identity/v3/projects/25e832f49fbc4fee98bbdf0334bd68af/users/f05b30c787cc4de68fd725600842d0ca/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:07:20.773330 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56aa0a7-d6d6-4b14-bdbe-31d11cc371cf None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:20.773621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56aa0a7-d6d6-4b14-bdbe-31d11cc371cf None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:20.773897 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56aa0a7-d6d6-4b14-bdbe-31d11cc371cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:20.805210 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-e56aa0a7-d6d6-4b14-bdbe-31d11cc371cf None None] MFA Rules not processed for user `f05b30c787cc4de68fd725600842d0ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:20.806432 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e56aa0a7-d6d6-4b14-bdbe-31d11cc371cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:20.892419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1122/2266] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:07:20.896811 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a8d47-74e9-41d6-8833-ac67b7272c43 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:20.917911 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:20.959055 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a8d47-74e9-41d6-8833-ac67b7272c43 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:20.963750 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bdb9d3-b991-4762-bd43-7b6fa6bab42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:20.964459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bdb9d3-b991-4762-bd43-7b6fa6bab42d admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:20.964739 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bdb9d3-b991-4762-bd43-7b6fa6bab42d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:20.965010 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bdb9d3-b991-4762-bd43-7b6fa6bab42d admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:20.965641 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bdb9d3-b991-4762-bd43-7b6fa6bab42d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:20.967536 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bdb9d3-b991-4762-bd43-7b6fa6bab42d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:21.071291 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1145/2267] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:07:20 2022] POST /identity/v3/projects => generated 385 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:07:21.075704 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56aa0a7-d6d6-4b14-bdbe-31d11cc371cf None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:21.086813 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56aa0a7-d6d6-4b14-bdbe-31d11cc371cf None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:21.099326 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:21.146206 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be2f29f-2d0e-4ea6-8d67-bc541fce7f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:21.198442 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2f29f-2d0e-4ea6-8d67-bc541fce7f9d admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:21.198442 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2f29f-2d0e-4ea6-8d67-bc541fce7f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:21.198442 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2f29f-2d0e-4ea6-8d67-bc541fce7f9d admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:21.198442 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be2f29f-2d0e-4ea6-8d67-bc541fce7f9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:21.198442 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be2f29f-2d0e-4ea6-8d67-bc541fce7f9d admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:21.385997 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1123/2268] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:07:21 2022] POST /identity/v3/users => generated 293 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:07:21.387266 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bdb9d3-b991-4762-bd43-7b6fa6bab42d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:21.397823 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bdb9d3-b991-4762-bd43-7b6fa6bab42d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:21.402619 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e7f17c-c3ed-49fb-b31e-eeb83a6c017b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:21.403241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7f17c-c3ed-49fb-b31e-eeb83a6c017b admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:21.403575 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7f17c-c3ed-49fb-b31e-eeb83a6c017b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:21.403871 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7f17c-c3ed-49fb-b31e-eeb83a6c017b admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:21.404582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e7f17c-c3ed-49fb-b31e-eeb83a6c017b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:21.406295 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e7f17c-c3ed-49fb-b31e-eeb83a6c017b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:21.418359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1146/2269] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:07:21.422492 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be2f29f-2d0e-4ea6-8d67-bc541fce7f9d admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:21.433152 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be2f29f-2d0e-4ea6-8d67-bc541fce7f9d admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:21.437841 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4142bf59-4512-45e2-8494-e5308ab3f2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:21.438462 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142bf59-4512-45e2-8494-e5308ab3f2c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:21.438741 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142bf59-4512-45e2-8494-e5308ab3f2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:21.439016 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142bf59-4512-45e2-8494-e5308ab3f2c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:21.439679 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4142bf59-4512-45e2-8494-e5308ab3f2c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:21.441338 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4142bf59-4512-45e2-8494-e5308ab3f2c7 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:21.451989 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1124/2270] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:07:21.456225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e7f17c-c3ed-49fb-b31e-eeb83a6c017b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:21.468994 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e7f17c-c3ed-49fb-b31e-eeb83a6c017b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:21.473683 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa6dcf-decd-4c7f-9ca3-24d4ee7f4ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:21.474364 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa6dcf-decd-4c7f-9ca3-24d4ee7f4ab1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:21.475127 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa6dcf-decd-4c7f-9ca3-24d4ee7f4ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:21.475406 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa6dcf-decd-4c7f-9ca3-24d4ee7f4ab1 admin admin] PATH_INFO: `/v3/projects/7e0ba230d4dd4040aad8e7d49b818f69/users/02efe9b8991a42b1b8da65dd5d786512/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:21.491790 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa6dcf-decd-4c7f-9ca3-24d4ee7f4ab1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e0ba230d4dd4040aad8e7d49b818f69, user_id=02efe9b8991a42b1b8da65dd5d786512, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:21.495174 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa6dcf-decd-4c7f-9ca3-24d4ee7f4ab1 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:21.559258 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1147/2271] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:07:21 2022] PUT /identity/v3/projects/7e0ba230d4dd4040aad8e7d49b818f69/users/02efe9b8991a42b1b8da65dd5d786512/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:07:21.569448 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21529d4a-8bee-429c-8171-19ff1a1608d4 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:21.569731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21529d4a-8bee-429c-8171-19ff1a1608d4 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:21.570004 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21529d4a-8bee-429c-8171-19ff1a1608d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:21.597939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-21529d4a-8bee-429c-8171-19ff1a1608d4 None None] MFA Rules not processed for user `02efe9b8991a42b1b8da65dd5d786512`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:21.599135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-21529d4a-8bee-429c-8171-19ff1a1608d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:21.668048 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1125/2272] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:07:21.673118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa6dcf-decd-4c7f-9ca3-24d4ee7f4ab1 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:21.739267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa6dcf-decd-4c7f-9ca3-24d4ee7f4ab1 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:21.744220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa2ac0-317b-423e-8e05-468cf46c8c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:21.744863 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa2ac0-317b-423e-8e05-468cf46c8c06 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:21.745143 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa2ac0-317b-423e-8e05-468cf46c8c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:21.745551 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa2ac0-317b-423e-8e05-468cf46c8c06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:21.746045 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afa2ac0-317b-423e-8e05-468cf46c8c06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:21.748052 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afa2ac0-317b-423e-8e05-468cf46c8c06 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:21.857483 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1148/2273] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:07:21 2022] POST /identity/v3/projects => generated 385 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:07:21.860653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21529d4a-8bee-429c-8171-19ff1a1608d4 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:21.871737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21529d4a-8bee-429c-8171-19ff1a1608d4 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:21.944171 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d38a07-89f8-4267-9dbb-271bfc6a615a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:21.944171 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d38a07-89f8-4267-9dbb-271bfc6a615a admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:21.944171 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d38a07-89f8-4267-9dbb-271bfc6a615a admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:21.944171 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d38a07-89f8-4267-9dbb-271bfc6a615a admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:21.944937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d38a07-89f8-4267-9dbb-271bfc6a615a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:21.944937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d38a07-89f8-4267-9dbb-271bfc6a615a admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:22.113956 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1126/2274] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:07:21 2022] POST /identity/v3/users => generated 299 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:07:22.118197 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa2ac0-317b-423e-8e05-468cf46c8c06 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:22.129151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa2ac0-317b-423e-8e05-468cf46c8c06 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:22.134400 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c40a7-f555-45fc-a978-25966a5a7025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:22.135078 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c40a7-f555-45fc-a978-25966a5a7025 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:22.135390 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c40a7-f555-45fc-a978-25966a5a7025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:22.135768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c40a7-f555-45fc-a978-25966a5a7025 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:22.136477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36c40a7-f555-45fc-a978-25966a5a7025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:22.138137 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36c40a7-f555-45fc-a978-25966a5a7025 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:22.149673 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1149/2275] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:07:22.154220 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d38a07-89f8-4267-9dbb-271bfc6a615a admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:22.164414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d38a07-89f8-4267-9dbb-271bfc6a615a admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:22.170200 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7dc1ff-cf68-48cd-a4e2-38d8a62db115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:22.170200 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7dc1ff-cf68-48cd-a4e2-38d8a62db115 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:22.170200 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7dc1ff-cf68-48cd-a4e2-38d8a62db115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:22.170504 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7dc1ff-cf68-48cd-a4e2-38d8a62db115 admin admin] PATH_INFO: `/v3/projects/aeca806820674f8fbea9b30eb213fd07/users/67466a0d3d324675b0290fb2ef627d9e/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:22.182951 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:22.186046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7dc1ff-cf68-48cd-a4e2-38d8a62db115 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeca806820674f8fbea9b30eb213fd07, user_id=67466a0d3d324675b0290fb2ef627d9e, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:22.188925 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7dc1ff-cf68-48cd-a4e2-38d8a62db115 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:22.264303 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1127/2276] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:07:22 2022] PUT /identity/v3/projects/aeca806820674f8fbea9b30eb213fd07/users/67466a0d3d324675b0290fb2ef627d9e/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:07:22.268335 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c40a7-f555-45fc-a978-25966a5a7025 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:22.278796 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c40a7-f555-45fc-a978-25966a5a7025 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:22.321956 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:22.336395 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f1cdf-14d6-4f68-8fca-52ed1a0c3ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:22.338543 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f1cdf-14d6-4f68-8fca-52ed1a0c3ed9 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:22.338543 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f1cdf-14d6-4f68-8fca-52ed1a0c3ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:22.338543 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f1cdf-14d6-4f68-8fca-52ed1a0c3ed9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:22.338543 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f1cdf-14d6-4f68-8fca-52ed1a0c3ed9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:22.339840 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f1cdf-14d6-4f68-8fca-52ed1a0c3ed9 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:22.351393 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1150/2277] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:07:22.358810 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7dc1ff-cf68-48cd-a4e2-38d8a62db115 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:22.370465 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7dc1ff-cf68-48cd-a4e2-38d8a62db115 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:22.375461 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dedddaa-e9bd-4cf2-846c-bf5670d05704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:22.381699 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dedddaa-e9bd-4cf2-846c-bf5670d05704 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:22.381699 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dedddaa-e9bd-4cf2-846c-bf5670d05704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:22.381699 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dedddaa-e9bd-4cf2-846c-bf5670d05704 admin admin] PATH_INFO: `/v3/projects/aeca806820674f8fbea9b30eb213fd07/users/67466a0d3d324675b0290fb2ef627d9e/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:22.381699 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dedddaa-e9bd-4cf2-846c-bf5670d05704 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeca806820674f8fbea9b30eb213fd07, user_id=67466a0d3d324675b0290fb2ef627d9e, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:22.382083 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dedddaa-e9bd-4cf2-846c-bf5670d05704 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:22.392532 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1128/2278] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:07:22 2022] PUT /identity/v3/projects/aeca806820674f8fbea9b30eb213fd07/users/67466a0d3d324675b0290fb2ef627d9e/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:07:22.402721 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d649a-3c2f-4af8-b306-46b85c4d87cc None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:22.403236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d649a-3c2f-4af8-b306-46b85c4d87cc None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:22.403357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d649a-3c2f-4af8-b306-46b85c4d87cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:22.431973 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-de8d649a-3c2f-4af8-b306-46b85c4d87cc None None] MFA Rules not processed for user `67466a0d3d324675b0290fb2ef627d9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:22.433260 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-de8d649a-3c2f-4af8-b306-46b85c4d87cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:22.506965 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1151/2279] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:22 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:07:22.520058 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dedddaa-e9bd-4cf2-846c-bf5670d05704 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:22.521706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-4dedddaa-e9bd-4cf2-846c-bf5670d05704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:22.583753 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dedddaa-e9bd-4cf2-846c-bf5670d05704 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:22.588460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67abed4-2f18-4d43-975f-8db9c83c4eca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:22.589622 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67abed4-2f18-4d43-975f-8db9c83c4eca service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:22.589898 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67abed4-2f18-4d43-975f-8db9c83c4eca service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:22.590155 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67abed4-2f18-4d43-975f-8db9c83c4eca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:22.651555 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67abed4-2f18-4d43-975f-8db9c83c4eca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:22.653185 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67abed4-2f18-4d43-975f-8db9c83c4eca service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:22.662750 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1129/2280] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:07:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:07:23.407281 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d649a-3c2f-4af8-b306-46b85c4d87cc None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:23.409268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-de8d649a-3c2f-4af8-b306-46b85c4d87cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:23.511449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d649a-3c2f-4af8-b306-46b85c4d87cc None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:23.516525 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e0224-8c92-4d9a-891e-694b0dc85d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:23.517417 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e0224-8c92-4d9a-891e-694b0dc85d86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:23.517759 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e0224-8c92-4d9a-891e-694b0dc85d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:23.518217 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e0224-8c92-4d9a-891e-694b0dc85d86 admin admin] PATH_INFO: `/v3/users/efa09c2f895145e7b6f2978d0cc3a3d8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:23.520112 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92e0224-8c92-4d9a-891e-694b0dc85d86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efa09c2f895145e7b6f2978d0cc3a3d8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:23.524215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92e0224-8c92-4d9a-891e-694b0dc85d86 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:23.695490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e92e0224-8c92-4d9a-891e-694b0dc85d86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efa09c2f895145e7b6f2978d0cc3a3d8'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:23.696340 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e92e0224-8c92-4d9a-891e-694b0dc85d86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efa09c2f895145e7b6f2978d0cc3a3d8'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:23.697078 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e92e0224-8c92-4d9a-891e-694b0dc85d86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efa09c2f895145e7b6f2978d0cc3a3d8'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:23.711550 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e92e0224-8c92-4d9a-891e-694b0dc85d86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efa09c2f895145e7b6f2978d0cc3a3d8'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:23.721044 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:23.726011 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e92e0224-8c92-4d9a-891e-694b0dc85d86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efa09c2f895145e7b6f2978d0cc3a3d8'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:23.811753 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1152/2281] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:07:23 2022] DELETE /identity/v3/users/efa09c2f895145e7b6f2978d0cc3a3d8 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:07:23.816156 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d45c64-be25-46d0-a357-14a43c8c1057 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:23.816647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d45c64-be25-46d0-a357-14a43c8c1057 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:23.816728 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d45c64-be25-46d0-a357-14a43c8c1057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:23.846872 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-68d45c64-be25-46d0-a357-14a43c8c1057 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:23.848174 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-68d45c64-be25-46d0-a357-14a43c8c1057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:23.925358 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1130/2282] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:07:23.938163 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e0224-8c92-4d9a-891e-694b0dc85d86 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:23.938914 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e92e0224-8c92-4d9a-891e-694b0dc85d86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:24.005622 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e0224-8c92-4d9a-891e-694b0dc85d86 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:24.015443 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad17ad-89f4-4330-a54a-04f07e7922d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:24.016116 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad17ad-89f4-4330-a54a-04f07e7922d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:24.016402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad17ad-89f4-4330-a54a-04f07e7922d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:24.016674 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad17ad-89f4-4330-a54a-04f07e7922d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:24.027075 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ad17ad-89f4-4330-a54a-04f07e7922d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:24.028613 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ad17ad-89f4-4330-a54a-04f07e7922d7 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:24.039306 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1153/2283] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:07:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:07:24.478331 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d45c64-be25-46d0-a357-14a43c8c1057 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:24.480633 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-68d45c64-be25-46d0-a357-14a43c8c1057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:24.494538 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d45c64-be25-46d0-a357-14a43c8c1057 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:24.499322 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cac39d-c705-49c0-b582-4f867ff123bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:24.502517 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cac39d-c705-49c0-b582-4f867ff123bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:24.502517 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cac39d-c705-49c0-b582-4f867ff123bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:24.502517 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cac39d-c705-49c0-b582-4f867ff123bd admin admin] PATH_INFO: `/v3/projects/eececfc64fc34822815d599124b70490` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:24.502517 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cac39d-c705-49c0-b582-4f867ff123bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=eececfc64fc34822815d599124b70490)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:24.505385 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cac39d-c705-49c0-b582-4f867ff123bd admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:24.617399 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:24.629526 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-d8cac39d-c705-49c0-b582-4f867ff123bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eececfc64fc34822815d599124b70490', 'request_id': 'req-d8cac39d-c705-49c0-b582-4f867ff123bd'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:24.716652 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-d8cac39d-c705-49c0-b582-4f867ff123bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eececfc64fc34822815d599124b70490', 'request_id': 'req-d8cac39d-c705-49c0-b582-4f867ff123bd'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:24.724064 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-d8cac39d-c705-49c0-b582-4f867ff123bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eececfc64fc34822815d599124b70490', 'request_id': 'req-d8cac39d-c705-49c0-b582-4f867ff123bd'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:24.744281 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-d8cac39d-c705-49c0-b582-4f867ff123bd admin admin] The token cache is being invalidate because project eececfc64fc34822815d599124b70490 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:07:24.744589 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-d8cac39d-c705-49c0-b582-4f867ff123bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:24.745320 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-d8cac39d-c705-49c0-b582-4f867ff123bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:24.758001 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1131/2284] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:07:24 2022] DELETE /identity/v3/projects/eececfc64fc34822815d599124b70490 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:07:24.781235 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad17ad-89f4-4330-a54a-04f07e7922d7 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:24.783638 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e5ad17ad-89f4-4330-a54a-04f07e7922d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:24.901521 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad17ad-89f4-4330-a54a-04f07e7922d7 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:24.970319 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d54f92-e4d1-4370-8a44-ad62f8ac0191 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:24.971382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d54f92-e4d1-4370-8a44-ad62f8ac0191 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:24.971819 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d54f92-e4d1-4370-8a44-ad62f8ac0191 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:24.972438 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d54f92-e4d1-4370-8a44-ad62f8ac0191 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:24.974045 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e4d54f92-e4d1-4370-8a44-ad62f8ac0191 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:25.057183 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d54f92-e4d1-4370-8a44-ad62f8ac0191 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:25.058819 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d54f92-e4d1-4370-8a44-ad62f8ac0191 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:25.062460 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:25.069168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1154/2285] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:07:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:07:26.874939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cac39d-c705-49c0-b582-4f867ff123bd admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:26.877081 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d8cac39d-c705-49c0-b582-4f867ff123bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:26.890529 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cac39d-c705-49c0-b582-4f867ff123bd admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:26.894119 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ad318-dbd3-455c-9e51-f30e93763e9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:26.894723 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ad318-dbd3-455c-9e51-f30e93763e9c service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:26.895008 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ad318-dbd3-455c-9e51-f30e93763e9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:26.895284 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ad318-dbd3-455c-9e51-f30e93763e9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:26.897126 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d45ad318-dbd3-455c-9e51-f30e93763e9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:26.974908 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45ad318-dbd3-455c-9e51-f30e93763e9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:26.976414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45ad318-dbd3-455c-9e51-f30e93763e9c service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:26.990800 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1132/2286] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:07:26 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:07:31.792379 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d54f92-e4d1-4370-8a44-ad62f8ac0191 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:31.794092 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e4d54f92-e4d1-4370-8a44-ad62f8ac0191 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:31.863288 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d54f92-e4d1-4370-8a44-ad62f8ac0191 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:31.868165 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd709c4-7e24-4bdd-8876-5e54f714dbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:31.868801 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd709c4-7e24-4bdd-8876-5e54f714dbd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:31.869081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd709c4-7e24-4bdd-8876-5e54f714dbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:31.869355 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd709c4-7e24-4bdd-8876-5e54f714dbd9 admin admin] PATH_INFO: `/v3/users/399677d2da32463fa2389b38a146068a` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:31.870481 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd709c4-7e24-4bdd-8876-5e54f714dbd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=399677d2da32463fa2389b38a146068a)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:31.872532 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd709c4-7e24-4bdd-8876-5e54f714dbd9 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:32.090527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fbd709c4-7e24-4bdd-8876-5e54f714dbd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '399677d2da32463fa2389b38a146068a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:32.090903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fbd709c4-7e24-4bdd-8876-5e54f714dbd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '399677d2da32463fa2389b38a146068a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:32.091590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fbd709c4-7e24-4bdd-8876-5e54f714dbd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '399677d2da32463fa2389b38a146068a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:32.099481 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fbd709c4-7e24-4bdd-8876-5e54f714dbd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '399677d2da32463fa2389b38a146068a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:32.112195 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fbd709c4-7e24-4bdd-8876-5e54f714dbd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '399677d2da32463fa2389b38a146068a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:32.193462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1155/2287] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:07:31 2022] DELETE /identity/v3/users/399677d2da32463fa2389b38a146068a => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:07:32.199341 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452df5b-a437-44da-a08d-64f42fbd852a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:32.200017 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452df5b-a437-44da-a08d-64f42fbd852a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:32.200469 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452df5b-a437-44da-a08d-64f42fbd852a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:32.227698 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-f452df5b-a437-44da-a08d-64f42fbd852a None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:32.228962 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f452df5b-a437-44da-a08d-64f42fbd852a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:32.291368 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1133/2288] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:07:32.301523 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd709c4-7e24-4bdd-8876-5e54f714dbd9 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:32.302947 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fbd709c4-7e24-4bdd-8876-5e54f714dbd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:32.368188 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd709c4-7e24-4bdd-8876-5e54f714dbd9 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:32.372922 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9212913-4312-4f50-849d-5244e0ee38b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:32.373534 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9212913-4312-4f50-849d-5244e0ee38b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:32.373812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9212913-4312-4f50-849d-5244e0ee38b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:32.374353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9212913-4312-4f50-849d-5244e0ee38b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:32.387211 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9212913-4312-4f50-849d-5244e0ee38b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:32.388881 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9212913-4312-4f50-849d-5244e0ee38b1 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:32.400269 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1156/2289] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:07:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:07:33.048773 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452df5b-a437-44da-a08d-64f42fbd852a None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:33.050318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f452df5b-a437-44da-a08d-64f42fbd852a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:33.064440 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452df5b-a437-44da-a08d-64f42fbd852a None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:33.069452 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf0ba5-54da-41ba-b3d4-6ebe68b8e0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:33.070755 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf0ba5-54da-41ba-b3d4-6ebe68b8e0d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:33.071098 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf0ba5-54da-41ba-b3d4-6ebe68b8e0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:33.071447 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf0ba5-54da-41ba-b3d4-6ebe68b8e0d4 admin admin] PATH_INFO: `/v3/users/f05b30c787cc4de68fd725600842d0ca` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:33.072670 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cf0ba5-54da-41ba-b3d4-6ebe68b8e0d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f05b30c787cc4de68fd725600842d0ca)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:33.074747 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cf0ba5-54da-41ba-b3d4-6ebe68b8e0d4 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:33.189836 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9212913-4312-4f50-849d-5244e0ee38b1 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:33.191347 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c9212913-4312-4f50-849d-5244e0ee38b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:33.204713 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9212913-4312-4f50-849d-5244e0ee38b1 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:33.209803 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec33d76-63c4-4c39-b812-b05cfc0abfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:33.209963 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec33d76-63c4-4c39-b812-b05cfc0abfb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:33.210104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec33d76-63c4-4c39-b812-b05cfc0abfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:33.210416 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec33d76-63c4-4c39-b812-b05cfc0abfb8 admin admin] PATH_INFO: `/v3/projects/9ca33fbf52a14fe3a25b3c8b77946524` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:33.211577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec33d76-63c4-4c39-b812-b05cfc0abfb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ca33fbf52a14fe3a25b3c8b77946524)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:33.213534 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec33d76-63c4-4c39-b812-b05cfc0abfb8 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:33.239147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:33.264015 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:33.266339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-62cf0ba5-54da-41ba-b3d4-6ebe68b8e0d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f05b30c787cc4de68fd725600842d0ca'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:33.267046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-62cf0ba5-54da-41ba-b3d4-6ebe68b8e0d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f05b30c787cc4de68fd725600842d0ca'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:33.277823 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-62cf0ba5-54da-41ba-b3d4-6ebe68b8e0d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f05b30c787cc4de68fd725600842d0ca'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:33.291050 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-62cf0ba5-54da-41ba-b3d4-6ebe68b8e0d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f05b30c787cc4de68fd725600842d0ca'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:33.292103 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-62cf0ba5-54da-41ba-b3d4-6ebe68b8e0d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f05b30c787cc4de68fd725600842d0ca'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:33.334185 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bec33d76-63c4-4c39-b812-b05cfc0abfb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ca33fbf52a14fe3a25b3c8b77946524', 'request_id': 'req-bec33d76-63c4-4c39-b812-b05cfc0abfb8'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:33.341606 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bec33d76-63c4-4c39-b812-b05cfc0abfb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ca33fbf52a14fe3a25b3c8b77946524', 'request_id': 'req-bec33d76-63c4-4c39-b812-b05cfc0abfb8'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:33.394619 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1134/2290] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:07:33 2022] DELETE /identity/v3/users/f05b30c787cc4de68fd725600842d0ca => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:07:33.399134 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4f276-deaf-4d82-ad2f-2e2247b2c441 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:33.399446 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4f276-deaf-4d82-ad2f-2e2247b2c441 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:33.399725 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4f276-deaf-4d82-ad2f-2e2247b2c441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:33.421714 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-f3f4f276-deaf-4d82-ad2f-2e2247b2c441 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:33.422941 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f3f4f276-deaf-4d82-ad2f-2e2247b2c441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:33.445883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bec33d76-63c4-4c39-b812-b05cfc0abfb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ca33fbf52a14fe3a25b3c8b77946524', 'request_id': 'req-bec33d76-63c4-4c39-b812-b05cfc0abfb8'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:33.457273 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bec33d76-63c4-4c39-b812-b05cfc0abfb8 admin admin] The token cache is being invalidate because project 9ca33fbf52a14fe3a25b3c8b77946524 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:07:33.463316 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bec33d76-63c4-4c39-b812-b05cfc0abfb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:33.463316 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bec33d76-63c4-4c39-b812-b05cfc0abfb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:33.463316 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1157/2291] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:07:33 2022] DELETE /identity/v3/projects/9ca33fbf52a14fe3a25b3c8b77946524 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:07:33.463606 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec33d76-63c4-4c39-b812-b05cfc0abfb8 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:33.464845 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bec33d76-63c4-4c39-b812-b05cfc0abfb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:33.484507 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1135/2292] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:07:33.494964 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f4f276-deaf-4d82-ad2f-2e2247b2c441 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:33.496324 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f3f4f276-deaf-4d82-ad2f-2e2247b2c441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:33.529028 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec33d76-63c4-4c39-b812-b05cfc0abfb8 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:33.533819 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28018440-8231-4b48-b98e-6508baed86e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:33.534446 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28018440-8231-4b48-b98e-6508baed86e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:33.534727 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28018440-8231-4b48-b98e-6508baed86e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:33.535001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28018440-8231-4b48-b98e-6508baed86e4 admin admin] PATH_INFO: `/v3/users/6111b47e53a34f2997f5b7ed7d4514bf` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:33.536163 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28018440-8231-4b48-b98e-6508baed86e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6111b47e53a34f2997f5b7ed7d4514bf)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:33.538086 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28018440-8231-4b48-b98e-6508baed86e4 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:33.554177 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f4f276-deaf-4d82-ad2f-2e2247b2c441 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:33.558193 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4e450-c915-4d01-a7e9-6fe0c6403263 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:33.558850 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4e450-c915-4d01-a7e9-6fe0c6403263 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:33.559127 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4e450-c915-4d01-a7e9-6fe0c6403263 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:33.559400 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4e450-c915-4d01-a7e9-6fe0c6403263 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:33.568559 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc4e450-c915-4d01-a7e9-6fe0c6403263 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:33.570105 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc4e450-c915-4d01-a7e9-6fe0c6403263 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:33.579649 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1136/2293] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:07:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:07:33.755248 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-28018440-8231-4b48-b98e-6508baed86e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6111b47e53a34f2997f5b7ed7d4514bf'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:33.755937 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-28018440-8231-4b48-b98e-6508baed86e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6111b47e53a34f2997f5b7ed7d4514bf'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:33.756632 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-28018440-8231-4b48-b98e-6508baed86e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6111b47e53a34f2997f5b7ed7d4514bf'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:33.763871 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-28018440-8231-4b48-b98e-6508baed86e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6111b47e53a34f2997f5b7ed7d4514bf'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:33.778518 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-28018440-8231-4b48-b98e-6508baed86e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6111b47e53a34f2997f5b7ed7d4514bf'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:33.880989 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1158/2294] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:07:33 2022] DELETE /identity/v3/users/6111b47e53a34f2997f5b7ed7d4514bf => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:07:34.059111 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4e450-c915-4d01-a7e9-6fe0c6403263 service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:34.060834 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8cc4e450-c915-4d01-a7e9-6fe0c6403263 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:34.082154 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4e450-c915-4d01-a7e9-6fe0c6403263 service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:34.144585 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f8e819-2923-41b8-bb93-17dc21e2aee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:34.145662 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f8e819-2923-41b8-bb93-17dc21e2aee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:34.145892 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f8e819-2923-41b8-bb93-17dc21e2aee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:34.146239 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f8e819-2923-41b8-bb93-17dc21e2aee1 admin admin] PATH_INFO: `/v3/projects/25e832f49fbc4fee98bbdf0334bd68af` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:34.147553 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f8e819-2923-41b8-bb93-17dc21e2aee1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25e832f49fbc4fee98bbdf0334bd68af)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:34.150164 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f8e819-2923-41b8-bb93-17dc21e2aee1 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:34.159361 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:34.299044 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-53f8e819-2923-41b8-bb93-17dc21e2aee1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25e832f49fbc4fee98bbdf0334bd68af', 'request_id': 'req-53f8e819-2923-41b8-bb93-17dc21e2aee1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:34.394311 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-53f8e819-2923-41b8-bb93-17dc21e2aee1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25e832f49fbc4fee98bbdf0334bd68af', 'request_id': 'req-53f8e819-2923-41b8-bb93-17dc21e2aee1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:34.401260 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-53f8e819-2923-41b8-bb93-17dc21e2aee1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25e832f49fbc4fee98bbdf0334bd68af', 'request_id': 'req-53f8e819-2923-41b8-bb93-17dc21e2aee1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:34.411900 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-53f8e819-2923-41b8-bb93-17dc21e2aee1 admin admin] The token cache is being invalidate because project 25e832f49fbc4fee98bbdf0334bd68af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:07:34.412274 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-53f8e819-2923-41b8-bb93-17dc21e2aee1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:34.412935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-53f8e819-2923-41b8-bb93-17dc21e2aee1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:34.414484 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1137/2295] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:07:34 2022] DELETE /identity/v3/projects/25e832f49fbc4fee98bbdf0334bd68af => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:07:34.418539 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28018440-8231-4b48-b98e-6508baed86e4 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:34.419946 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-28018440-8231-4b48-b98e-6508baed86e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:34.433477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28018440-8231-4b48-b98e-6508baed86e4 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:34.452123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:34.489938 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e2ee96-1a70-4617-960c-4a925f811cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:34.490591 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2ee96-1a70-4617-960c-4a925f811cf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:34.490876 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2ee96-1a70-4617-960c-4a925f811cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:34.491153 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2ee96-1a70-4617-960c-4a925f811cf1 admin admin] PATH_INFO: `/v3/users/02efe9b8991a42b1b8da65dd5d786512` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:34.493099 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e2ee96-1a70-4617-960c-4a925f811cf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02efe9b8991a42b1b8da65dd5d786512)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:34.493858 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f8e819-2923-41b8-bb93-17dc21e2aee1 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:34.495060 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e2ee96-1a70-4617-960c-4a925f811cf1 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:34.502511 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-53f8e819-2923-41b8-bb93-17dc21e2aee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:34.519771 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f8e819-2923-41b8-bb93-17dc21e2aee1 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:34.524730 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b2fc6-cc7c-4dce-ac12-dca84766e4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:34.525443 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b2fc6-cc7c-4dce-ac12-dca84766e4e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:34.525725 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b2fc6-cc7c-4dce-ac12-dca84766e4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:34.525997 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b2fc6-cc7c-4dce-ac12-dca84766e4e0 admin admin] PATH_INFO: `/v3/projects/719ee356a279427f94775e069471481c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:34.527267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878b2fc6-cc7c-4dce-ac12-dca84766e4e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=719ee356a279427f94775e069471481c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:34.529296 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878b2fc6-cc7c-4dce-ac12-dca84766e4e0 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:34.768864 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f8e2ee96-1a70-4617-960c-4a925f811cf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02efe9b8991a42b1b8da65dd5d786512'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:34.769107 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-878b2fc6-cc7c-4dce-ac12-dca84766e4e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '719ee356a279427f94775e069471481c', 'request_id': 'req-878b2fc6-cc7c-4dce-ac12-dca84766e4e0'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:34.769524 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f8e2ee96-1a70-4617-960c-4a925f811cf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02efe9b8991a42b1b8da65dd5d786512'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:34.770194 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f8e2ee96-1a70-4617-960c-4a925f811cf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02efe9b8991a42b1b8da65dd5d786512'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:34.777273 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f8e2ee96-1a70-4617-960c-4a925f811cf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02efe9b8991a42b1b8da65dd5d786512'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:34.789432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f8e2ee96-1a70-4617-960c-4a925f811cf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02efe9b8991a42b1b8da65dd5d786512'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:34.851950 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-878b2fc6-cc7c-4dce-ac12-dca84766e4e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '719ee356a279427f94775e069471481c', 'request_id': 'req-878b2fc6-cc7c-4dce-ac12-dca84766e4e0'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:34.859063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-878b2fc6-cc7c-4dce-ac12-dca84766e4e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '719ee356a279427f94775e069471481c', 'request_id': 'req-878b2fc6-cc7c-4dce-ac12-dca84766e4e0'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:34.869509 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-878b2fc6-cc7c-4dce-ac12-dca84766e4e0 admin admin] The token cache is being invalidate because project 719ee356a279427f94775e069471481c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:07:34.869801 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-878b2fc6-cc7c-4dce-ac12-dca84766e4e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:34.870433 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-878b2fc6-cc7c-4dce-ac12-dca84766e4e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:34.872051 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1138/2296] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:07:34 2022] DELETE /identity/v3/projects/719ee356a279427f94775e069471481c => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:07:34.877112 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c086379-3e18-4593-9c57-1af45797da57 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:34.877394 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c086379-3e18-4593-9c57-1af45797da57 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:34.877664 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c086379-3e18-4593-9c57-1af45797da57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:34.902332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-1c086379-3e18-4593-9c57-1af45797da57 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:34.903555 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-1c086379-3e18-4593-9c57-1af45797da57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:34.932764 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1159/2297] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:07:34 2022] DELETE /identity/v3/users/02efe9b8991a42b1b8da65dd5d786512 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:07:34.971079 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1139/2298] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:07:34.979052 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05dab1-5498-4043-b82b-cb97a1a0c30b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:34.979423 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05dab1-5498-4043-b82b-cb97a1a0c30b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:34.979706 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05dab1-5498-4043-b82b-cb97a1a0c30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:35.036666 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-0a05dab1-5498-4043-b82b-cb97a1a0c30b None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:35.037858 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0a05dab1-5498-4043-b82b-cb97a1a0c30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:35.113576 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1160/2299] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:07:35.118593 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c086379-3e18-4593-9c57-1af45797da57 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:35.131498 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c086379-3e18-4593-9c57-1af45797da57 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:35.140218 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864e48d-82a9-451a-a3c5-1e8f27f465d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:35.140939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864e48d-82a9-451a-a3c5-1e8f27f465d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:35.141261 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864e48d-82a9-451a-a3c5-1e8f27f465d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:35.141584 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864e48d-82a9-451a-a3c5-1e8f27f465d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:35.142363 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864e48d-82a9-451a-a3c5-1e8f27f465d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:07:35.142678 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864e48d-82a9-451a-a3c5-1e8f27f465d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:35.145005 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864e48d-82a9-451a-a3c5-1e8f27f465d9 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:35.162709 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1140/2300] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:07:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:07:35.164584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05dab1-5498-4043-b82b-cb97a1a0c30b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:35.197540 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05dab1-5498-4043-b82b-cb97a1a0c30b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:35.209909 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca9478-0ef5-490b-8173-9bcd2d78dfbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:35.210695 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca9478-0ef5-490b-8173-9bcd2d78dfbe admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:35.211015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca9478-0ef5-490b-8173-9bcd2d78dfbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:35.211243 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca9478-0ef5-490b-8173-9bcd2d78dfbe admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:35.211950 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca9478-0ef5-490b-8173-9bcd2d78dfbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:35.214005 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca9478-0ef5-490b-8173-9bcd2d78dfbe admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:35.340108 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1161/2301] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:07:35 2022] POST /identity/v3/projects => generated 365 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:07:35.342953 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864e48d-82a9-451a-a3c5-1e8f27f465d9 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:35.353200 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864e48d-82a9-451a-a3c5-1e8f27f465d9 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:35.356838 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:35.414974 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696235f-9dd3-4975-a305-d4ce7415a90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:35.415576 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696235f-9dd3-4975-a305-d4ce7415a90a admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:35.415881 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696235f-9dd3-4975-a305-d4ce7415a90a admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:35.416189 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696235f-9dd3-4975-a305-d4ce7415a90a admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:35.416814 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c696235f-9dd3-4975-a305-d4ce7415a90a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:35.418738 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c696235f-9dd3-4975-a305-d4ce7415a90a admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:35.442728 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca9478-0ef5-490b-8173-9bcd2d78dfbe admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:35.444540 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4fca9478-0ef5-490b-8173-9bcd2d78dfbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:35.457451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca9478-0ef5-490b-8173-9bcd2d78dfbe admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:35.462097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e79bf-7d41-48a5-a3ea-b9422111e4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:35.462785 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e79bf-7d41-48a5-a3ea-b9422111e4a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:35.463061 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e79bf-7d41-48a5-a3ea-b9422111e4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:35.463335 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e79bf-7d41-48a5-a3ea-b9422111e4a0 admin admin] PATH_INFO: `/v3/projects/7e0ba230d4dd4040aad8e7d49b818f69` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:35.464478 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e79bf-7d41-48a5-a3ea-b9422111e4a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e0ba230d4dd4040aad8e7d49b818f69)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:35.466383 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e79bf-7d41-48a5-a3ea-b9422111e4a0 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:35.483945 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:35.508940 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1141/2302] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:07:35 2022] POST /identity/v3/users => generated 283 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:07:35.513105 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696235f-9dd3-4975-a305-d4ce7415a90a admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:35.522492 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696235f-9dd3-4975-a305-d4ce7415a90a admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:35.527825 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d742b45-0dbb-4504-b679-fcfc0d2bbcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:35.528550 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d742b45-0dbb-4504-b679-fcfc0d2bbcca admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:35.528833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d742b45-0dbb-4504-b679-fcfc0d2bbcca admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:35.529102 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d742b45-0dbb-4504-b679-fcfc0d2bbcca admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:35.529735 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d742b45-0dbb-4504-b679-fcfc0d2bbcca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:35.531294 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d742b45-0dbb-4504-b679-fcfc0d2bbcca admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:35.543288 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1142/2303] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:07:35.547103 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d742b45-0dbb-4504-b679-fcfc0d2bbcca admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:35.557237 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d742b45-0dbb-4504-b679-fcfc0d2bbcca admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:35.561779 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8872221-6358-4b55-8e7a-e75875dc55b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:35.562354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8872221-6358-4b55-8e7a-e75875dc55b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:35.562626 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8872221-6358-4b55-8e7a-e75875dc55b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:35.564057 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8872221-6358-4b55-8e7a-e75875dc55b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:35.564057 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8872221-6358-4b55-8e7a-e75875dc55b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:35.565176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8872221-6358-4b55-8e7a-e75875dc55b6 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:35.572540 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-634e79bf-7d41-48a5-a3ea-b9422111e4a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e0ba230d4dd4040aad8e7d49b818f69', 'request_id': 'req-634e79bf-7d41-48a5-a3ea-b9422111e4a0'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:35.574795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1143/2304] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:07:35.578994 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8872221-6358-4b55-8e7a-e75875dc55b6 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:35.580101 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-634e79bf-7d41-48a5-a3ea-b9422111e4a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e0ba230d4dd4040aad8e7d49b818f69', 'request_id': 'req-634e79bf-7d41-48a5-a3ea-b9422111e4a0'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:35.588710 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8872221-6358-4b55-8e7a-e75875dc55b6 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:35.643411 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad0308-4709-44ca-a381-31966db99142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:35.644159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad0308-4709-44ca-a381-31966db99142 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:35.644372 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad0308-4709-44ca-a381-31966db99142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:35.644645 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad0308-4709-44ca-a381-31966db99142 admin admin] PATH_INFO: `/v3/projects/a5ead2e7d5e5497bbe48196ef6d1acad/users/8b179d722d9e4dac980a3c7b88c5fa0e/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:35.651622 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-634e79bf-7d41-48a5-a3ea-b9422111e4a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e0ba230d4dd4040aad8e7d49b818f69', 'request_id': 'req-634e79bf-7d41-48a5-a3ea-b9422111e4a0'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:35.658828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ad0308-4709-44ca-a381-31966db99142 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5ead2e7d5e5497bbe48196ef6d1acad, user_id=8b179d722d9e4dac980a3c7b88c5fa0e, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:35.661524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ad0308-4709-44ca-a381-31966db99142 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:35.663108 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-634e79bf-7d41-48a5-a3ea-b9422111e4a0 admin admin] The token cache is being invalidate because project 7e0ba230d4dd4040aad8e7d49b818f69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:07:35.663909 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-634e79bf-7d41-48a5-a3ea-b9422111e4a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:35.664593 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-634e79bf-7d41-48a5-a3ea-b9422111e4a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:35.666102 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1162/2305] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:07:35 2022] DELETE /identity/v3/projects/7e0ba230d4dd4040aad8e7d49b818f69 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:07:35.669974 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e79bf-7d41-48a5-a3ea-b9422111e4a0 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:35.671342 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-634e79bf-7d41-48a5-a3ea-b9422111e4a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:35.684575 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e79bf-7d41-48a5-a3ea-b9422111e4a0 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:35.689195 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf419103-a883-49b3-ac1f-96866b027c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:35.689833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf419103-a883-49b3-ac1f-96866b027c79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:35.690193 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf419103-a883-49b3-ac1f-96866b027c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:35.690470 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf419103-a883-49b3-ac1f-96866b027c79 admin admin] PATH_INFO: `/v3/users/67466a0d3d324675b0290fb2ef627d9e` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:35.691635 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf419103-a883-49b3-ac1f-96866b027c79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67466a0d3d324675b0290fb2ef627d9e)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:35.693623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf419103-a883-49b3-ac1f-96866b027c79 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:35.732359 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1144/2306] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:07:35 2022] PUT /identity/v3/projects/a5ead2e7d5e5497bbe48196ef6d1acad/users/8b179d722d9e4dac980a3c7b88c5fa0e/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:07:35.744294 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba59a9ca-e7b8-47f4-8fa6-1a14bface964 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:35.744579 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba59a9ca-e7b8-47f4-8fa6-1a14bface964 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:35.744870 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba59a9ca-e7b8-47f4-8fa6-1a14bface964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:35.771321 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-ba59a9ca-e7b8-47f4-8fa6-1a14bface964 None None] MFA Rules not processed for user `8b179d722d9e4dac980a3c7b88c5fa0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:35.772558 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ba59a9ca-e7b8-47f4-8fa6-1a14bface964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:35.837109 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1145/2307] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:07:35.841429 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba59a9ca-e7b8-47f4-8fa6-1a14bface964 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:35.842732 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ba59a9ca-e7b8-47f4-8fa6-1a14bface964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:35.903883 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:35.906586 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba59a9ca-e7b8-47f4-8fa6-1a14bface964 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:35.910797 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-cf419103-a883-49b3-ac1f-96866b027c79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67466a0d3d324675b0290fb2ef627d9e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:35.911354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d3a61-d5f3-433f-964a-82ecf6fa0cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:35.911670 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-cf419103-a883-49b3-ac1f-96866b027c79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67466a0d3d324675b0290fb2ef627d9e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:35.911964 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d3a61-d5f3-433f-964a-82ecf6fa0cd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:35.912530 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-cf419103-a883-49b3-ac1f-96866b027c79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67466a0d3d324675b0290fb2ef627d9e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:35.912650 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d3a61-d5f3-433f-964a-82ecf6fa0cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:35.912817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d3a61-d5f3-433f-964a-82ecf6fa0cd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:35.914136 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d3a61-d5f3-433f-964a-82ecf6fa0cd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:35.916099 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d3a61-d5f3-433f-964a-82ecf6fa0cd7 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:35.921066 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-cf419103-a883-49b3-ac1f-96866b027c79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67466a0d3d324675b0290fb2ef627d9e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:35.933064 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-cf419103-a883-49b3-ac1f-96866b027c79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67466a0d3d324675b0290fb2ef627d9e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:36.024469 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1146/2308] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:07:35 2022] POST /identity/v3/projects => generated 365 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:07:36.028460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d3a61-d5f3-433f-964a-82ecf6fa0cd7 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:36.030055 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-9b5d3a61-d5f3-433f-964a-82ecf6fa0cd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:36.030458 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1163/2309] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:07:35 2022] DELETE /identity/v3/users/67466a0d3d324675b0290fb2ef627d9e => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:07:36.093744 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d3a61-d5f3-433f-964a-82ecf6fa0cd7 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:36.150245 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177adca4-5136-4e14-b70f-72362d74910c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:36.150854 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177adca4-5136-4e14-b70f-72362d74910c admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:36.151137 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177adca4-5136-4e14-b70f-72362d74910c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:36.151419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177adca4-5136-4e14-b70f-72362d74910c admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:36.152046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177adca4-5136-4e14-b70f-72362d74910c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:36.153996 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177adca4-5136-4e14-b70f-72362d74910c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:36.289551 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1147/2310] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:07:36 2022] POST /identity/v3/users => generated 289 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:07:36.291141 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf419103-a883-49b3-ac1f-96866b027c79 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:36.301127 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf419103-a883-49b3-ac1f-96866b027c79 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:36.305964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072ee09-1c0d-47f7-8c6c-aee722fed062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:36.306641 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072ee09-1c0d-47f7-8c6c-aee722fed062 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:36.307029 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072ee09-1c0d-47f7-8c6c-aee722fed062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:36.307394 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072ee09-1c0d-47f7-8c6c-aee722fed062 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:36.308162 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5072ee09-1c0d-47f7-8c6c-aee722fed062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:36.309942 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5072ee09-1c0d-47f7-8c6c-aee722fed062 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:36.321582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1164/2311] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:07:36.328452 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177adca4-5136-4e14-b70f-72362d74910c admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:36.339086 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177adca4-5136-4e14-b70f-72362d74910c admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:36.344397 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f61899-d09b-4b36-832b-7820bd03985b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:36.345003 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f61899-d09b-4b36-832b-7820bd03985b admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:36.345290 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f61899-d09b-4b36-832b-7820bd03985b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:36.345566 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f61899-d09b-4b36-832b-7820bd03985b admin admin] PATH_INFO: `/v3/projects/ad437d1de20940c6b3f0f94a64ce3074/users/6553fa301c954f12b0e1e31b32307371/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:36.360473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f61899-d09b-4b36-832b-7820bd03985b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad437d1de20940c6b3f0f94a64ce3074, user_id=6553fa301c954f12b0e1e31b32307371, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:36.363138 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f61899-d09b-4b36-832b-7820bd03985b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:36.430281 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072ee09-1c0d-47f7-8c6c-aee722fed062 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:36.432005 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5072ee09-1c0d-47f7-8c6c-aee722fed062 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:36.438446 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1148/2312] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:07:36 2022] PUT /identity/v3/projects/ad437d1de20940c6b3f0f94a64ce3074/users/6553fa301c954f12b0e1e31b32307371/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:07:36.442361 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f61899-d09b-4b36-832b-7820bd03985b admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:36.451094 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072ee09-1c0d-47f7-8c6c-aee722fed062 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:36.526982 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07868eb-4d86-4e66-80bd-e09cf0f89933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:36.527619 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07868eb-4d86-4e66-80bd-e09cf0f89933 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:36.527903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07868eb-4d86-4e66-80bd-e09cf0f89933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:36.528216 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07868eb-4d86-4e66-80bd-e09cf0f89933 admin admin] PATH_INFO: `/v3/projects/aeca806820674f8fbea9b30eb213fd07` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:36.529348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07868eb-4d86-4e66-80bd-e09cf0f89933 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aeca806820674f8fbea9b30eb213fd07)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:36.531314 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07868eb-4d86-4e66-80bd-e09cf0f89933 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:36.540040 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:36.542184 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f61899-d09b-4b36-832b-7820bd03985b admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:36.552041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2dc88e-86bb-4cdc-b98e-52ba3b462da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:36.552041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2dc88e-86bb-4cdc-b98e-52ba3b462da3 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:36.552041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2dc88e-86bb-4cdc-b98e-52ba3b462da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:36.552041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2dc88e-86bb-4cdc-b98e-52ba3b462da3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:36.552411 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2dc88e-86bb-4cdc-b98e-52ba3b462da3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:36.553113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2dc88e-86bb-4cdc-b98e-52ba3b462da3 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:36.565117 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1149/2313] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:07:36.569500 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2dc88e-86bb-4cdc-b98e-52ba3b462da3 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:36.577417 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:36.582709 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2dc88e-86bb-4cdc-b98e-52ba3b462da3 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:36.587635 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c19d8-596e-4858-9de5-5bac290a21eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:36.588320 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c19d8-596e-4858-9de5-5bac290a21eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:36.588612 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c19d8-596e-4858-9de5-5bac290a21eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:36.588885 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c19d8-596e-4858-9de5-5bac290a21eb admin admin] PATH_INFO: `/v3/projects/ad437d1de20940c6b3f0f94a64ce3074/users/6553fa301c954f12b0e1e31b32307371/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:36.590800 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c19d8-596e-4858-9de5-5bac290a21eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad437d1de20940c6b3f0f94a64ce3074, user_id=6553fa301c954f12b0e1e31b32307371, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:36.593495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c19d8-596e-4858-9de5-5bac290a21eb admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:36.604864 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1150/2314] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:07:36 2022] PUT /identity/v3/projects/ad437d1de20940c6b3f0f94a64ce3074/users/6553fa301c954f12b0e1e31b32307371/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:07:36.615083 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b0d3d-2cfa-4d2e-8eb6-2b96d209a57c None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:36.615373 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b0d3d-2cfa-4d2e-8eb6-2b96d209a57c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:36.615702 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b0d3d-2cfa-4d2e-8eb6-2b96d209a57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:36.645963 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-e31b0d3d-2cfa-4d2e-8eb6-2b96d209a57c None None] MFA Rules not processed for user `6553fa301c954f12b0e1e31b32307371`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:36.647243 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e31b0d3d-2cfa-4d2e-8eb6-2b96d209a57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:36.674040 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b07868eb-4d86-4e66-80bd-e09cf0f89933 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aeca806820674f8fbea9b30eb213fd07', 'request_id': 'req-b07868eb-4d86-4e66-80bd-e09cf0f89933'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:36.681866 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b07868eb-4d86-4e66-80bd-e09cf0f89933 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aeca806820674f8fbea9b30eb213fd07', 'request_id': 'req-b07868eb-4d86-4e66-80bd-e09cf0f89933'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:36.718109 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1151/2315] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:07:36.729607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b0d3d-2cfa-4d2e-8eb6-2b96d209a57c None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:36.730943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e31b0d3d-2cfa-4d2e-8eb6-2b96d209a57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:36.747642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b07868eb-4d86-4e66-80bd-e09cf0f89933 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aeca806820674f8fbea9b30eb213fd07', 'request_id': 'req-b07868eb-4d86-4e66-80bd-e09cf0f89933'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:36.759895 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b07868eb-4d86-4e66-80bd-e09cf0f89933 admin admin] The token cache is being invalidate because project aeca806820674f8fbea9b30eb213fd07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:07:36.760227 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b07868eb-4d86-4e66-80bd-e09cf0f89933 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:36.760885 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b07868eb-4d86-4e66-80bd-e09cf0f89933 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:36.762570 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1165/2316] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:07:36 2022] DELETE /identity/v3/projects/aeca806820674f8fbea9b30eb213fd07 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:07:36.767054 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582f104-abf9-4efe-83a4-19a1584e1f76 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:36.767339 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582f104-abf9-4efe-83a4-19a1584e1f76 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:36.767615 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582f104-abf9-4efe-83a4-19a1584e1f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:36.786780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5582f104-abf9-4efe-83a4-19a1584e1f76 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:36.787974 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5582f104-abf9-4efe-83a4-19a1584e1f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:36.789981 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b0d3d-2cfa-4d2e-8eb6-2b96d209a57c None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:36.839861 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed29928-c28b-4e23-b21d-9aa78372991d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:36.840510 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed29928-c28b-4e23-b21d-9aa78372991d service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:36.840789 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed29928-c28b-4e23-b21d-9aa78372991d service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:36.841060 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed29928-c28b-4e23-b21d-9aa78372991d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:36.842866 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5ed29928-c28b-4e23-b21d-9aa78372991d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:36.847569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1166/2317] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:07:36.855488 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74d062-9788-4ef1-81a1-fcbffbfcd8fe None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:36.855790 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74d062-9788-4ef1-81a1-fcbffbfcd8fe None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:36.856101 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74d062-9788-4ef1-81a1-fcbffbfcd8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:36.876013 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-be74d062-9788-4ef1-81a1-fcbffbfcd8fe None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:36.877229 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-be74d062-9788-4ef1-81a1-fcbffbfcd8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:36.887638 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1167/2318] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:07:36.891767 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74d062-9788-4ef1-81a1-fcbffbfcd8fe None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:36.900730 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74d062-9788-4ef1-81a1-fcbffbfcd8fe None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:36.905522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b2a755-a243-4a9d-bd83-20831c78d249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:36.905662 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed29928-c28b-4e23-b21d-9aa78372991d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:36.906101 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2a755-a243-4a9d-bd83-20831c78d249 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:36.906377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2a755-a243-4a9d-bd83-20831c78d249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:36.906985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2a755-a243-4a9d-bd83-20831c78d249 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:36.907088 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed29928-c28b-4e23-b21d-9aa78372991d service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:36.907577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b2a755-a243-4a9d-bd83-20831c78d249 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:07:36.907908 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b2a755-a243-4a9d-bd83-20831c78d249 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:36.909630 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b2a755-a243-4a9d-bd83-20831c78d249 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:36.917147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1152/2319] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:07:36 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:07:36.922082 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1168/2320] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:07:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:07:36.926519 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed29928-c28b-4e23-b21d-9aa78372991d service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:36.937391 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed29928-c28b-4e23-b21d-9aa78372991d service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:36.942106 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db45fff-c4ab-4354-89d7-5d6992660894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:36.942697 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db45fff-c4ab-4354-89d7-5d6992660894 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:36.942979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db45fff-c4ab-4354-89d7-5d6992660894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:36.943250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db45fff-c4ab-4354-89d7-5d6992660894 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:36.943887 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db45fff-c4ab-4354-89d7-5d6992660894 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:36.945912 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db45fff-c4ab-4354-89d7-5d6992660894 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:37.042192 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:37.049929 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1153/2321] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:07:36 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:07:37.054574 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b2a755-a243-4a9d-bd83-20831c78d249 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:37.064363 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b2a755-a243-4a9d-bd83-20831c78d249 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:37.119840 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a337dad1-8cd3-41b6-ad7d-f5a95fbb89bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:37.120407 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a337dad1-8cd3-41b6-ad7d-f5a95fbb89bd admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:37.120692 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a337dad1-8cd3-41b6-ad7d-f5a95fbb89bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:37.120969 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a337dad1-8cd3-41b6-ad7d-f5a95fbb89bd admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:37.121599 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a337dad1-8cd3-41b6-ad7d-f5a95fbb89bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:37.123528 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a337dad1-8cd3-41b6-ad7d-f5a95fbb89bd admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:37.162004 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db45fff-c4ab-4354-89d7-5d6992660894 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:37.164323 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5db45fff-c4ab-4354-89d7-5d6992660894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:37.220026 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db45fff-c4ab-4354-89d7-5d6992660894 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:37.225522 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687689b-8822-481e-b431-ee631758ed83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:37.226719 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687689b-8822-481e-b431-ee631758ed83 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:37.226928 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687689b-8822-481e-b431-ee631758ed83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:37.227195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687689b-8822-481e-b431-ee631758ed83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:37.228956 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-9687689b-8822-481e-b431-ee631758ed83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:37.235215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1169/2322] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:07:37 2022] POST /identity/v3/users => generated 287 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:07:37.235784 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a337dad1-8cd3-41b6-ad7d-f5a95fbb89bd admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:37.247918 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a337dad1-8cd3-41b6-ad7d-f5a95fbb89bd admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:37.252511 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:37.254887 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf3a52d-cec2-477e-bf06-747ef335998f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:37.255503 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf3a52d-cec2-477e-bf06-747ef335998f admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:37.255786 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf3a52d-cec2-477e-bf06-747ef335998f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:37.256058 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf3a52d-cec2-477e-bf06-747ef335998f admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:37.256729 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf3a52d-cec2-477e-bf06-747ef335998f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:37.258338 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf3a52d-cec2-477e-bf06-747ef335998f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:37.277435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1170/2323] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:07:37.281091 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf3a52d-cec2-477e-bf06-747ef335998f admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:37.292083 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf3a52d-cec2-477e-bf06-747ef335998f admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:37.296604 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687689b-8822-481e-b431-ee631758ed83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:37.296956 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a78a8-fe6a-4e04-b207-9983b4f3df84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:37.297608 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a78a8-fe6a-4e04-b207-9983b4f3df84 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:37.297954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a78a8-fe6a-4e04-b207-9983b4f3df84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:37.298189 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687689b-8822-481e-b431-ee631758ed83 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:37.298341 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a78a8-fe6a-4e04-b207-9983b4f3df84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:37.299043 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84a78a8-fe6a-4e04-b207-9983b4f3df84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:37.300718 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84a78a8-fe6a-4e04-b207-9983b4f3df84 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:37.307331 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1154/2324] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:07:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:07:37.314096 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1171/2325] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:07:37.315251 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687689b-8822-481e-b431-ee631758ed83 service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:37.326380 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687689b-8822-481e-b431-ee631758ed83 service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:37.332801 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad157365-c4ee-4576-b2aa-5082fd5d5b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:37.332801 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad157365-c4ee-4576-b2aa-5082fd5d5b96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:37.332801 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad157365-c4ee-4576-b2aa-5082fd5d5b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:37.332801 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad157365-c4ee-4576-b2aa-5082fd5d5b96 admin admin] PATH_INFO: `/v3/projects/287816ca60e74940a57777459509bf2e/users/50204a91e8b842048db190ebf66d0b7f/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:37.346929 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:37.353406 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad157365-c4ee-4576-b2aa-5082fd5d5b96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=287816ca60e74940a57777459509bf2e, user_id=50204a91e8b842048db190ebf66d0b7f, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:37.356107 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad157365-c4ee-4576-b2aa-5082fd5d5b96 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:37.435072 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1155/2326] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:07:37 2022] PUT /identity/v3/projects/287816ca60e74940a57777459509bf2e/users/50204a91e8b842048db190ebf66d0b7f/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:07:37.445110 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1050233-0efb-4871-96cb-3e2b2bd31117 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:37.445397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1050233-0efb-4871-96cb-3e2b2bd31117 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:37.445669 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1050233-0efb-4871-96cb-3e2b2bd31117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:37.474697 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a1050233-0efb-4871-96cb-3e2b2bd31117 None None] MFA Rules not processed for user `50204a91e8b842048db190ebf66d0b7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:37.476551 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a1050233-0efb-4871-96cb-3e2b2bd31117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:37.540298 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1172/2327] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:07:37.551581 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad157365-c4ee-4576-b2aa-5082fd5d5b96 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:37.553310 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ad157365-c4ee-4576-b2aa-5082fd5d5b96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:37.609378 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad157365-c4ee-4576-b2aa-5082fd5d5b96 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:37.613018 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae965c-f495-4dbe-b15e-b11eab882ecd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:37.613620 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae965c-f495-4dbe-b15e-b11eab882ecd service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:37.613900 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae965c-f495-4dbe-b15e-b11eab882ecd service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:37.614170 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae965c-f495-4dbe-b15e-b11eab882ecd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:37.622300 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae965c-f495-4dbe-b15e-b11eab882ecd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:37.623718 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae965c-f495-4dbe-b15e-b11eab882ecd service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:37.632939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1156/2328] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:07:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:07:37.657820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1050233-0efb-4871-96cb-3e2b2bd31117 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:37.745737 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1050233-0efb-4871-96cb-3e2b2bd31117 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:37.751145 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24dd056-6e3b-4a46-8984-add1105b3db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:37.751780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dd056-6e3b-4a46-8984-add1105b3db0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:37.752063 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dd056-6e3b-4a46-8984-add1105b3db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:37.755991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dd056-6e3b-4a46-8984-add1105b3db0 admin admin] PATH_INFO: `/v3/users/50204a91e8b842048db190ebf66d0b7f` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:37.755991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24dd056-6e3b-4a46-8984-add1105b3db0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50204a91e8b842048db190ebf66d0b7f)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:37.755991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24dd056-6e3b-4a46-8984-add1105b3db0 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:37.962975 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a24dd056-6e3b-4a46-8984-add1105b3db0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50204a91e8b842048db190ebf66d0b7f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:37.963625 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a24dd056-6e3b-4a46-8984-add1105b3db0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50204a91e8b842048db190ebf66d0b7f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:37.964257 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a24dd056-6e3b-4a46-8984-add1105b3db0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50204a91e8b842048db190ebf66d0b7f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:37.971689 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a24dd056-6e3b-4a46-8984-add1105b3db0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50204a91e8b842048db190ebf66d0b7f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:37.986094 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a24dd056-6e3b-4a46-8984-add1105b3db0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50204a91e8b842048db190ebf66d0b7f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:37.987845 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:38.089949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1173/2329] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:07:37 2022] DELETE /identity/v3/users/50204a91e8b842048db190ebf66d0b7f => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:07:38.094438 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299b330-93c4-4ef9-a691-e6712a0d06a5 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:38.094721 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299b330-93c4-4ef9-a691-e6712a0d06a5 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:38.095008 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299b330-93c4-4ef9-a691-e6712a0d06a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:38.113701 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-7299b330-93c4-4ef9-a691-e6712a0d06a5 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:38.114898 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-7299b330-93c4-4ef9-a691-e6712a0d06a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:38.171585 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1157/2330] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:07:38.181647 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24dd056-6e3b-4a46-8984-add1105b3db0 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:38.183086 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a24dd056-6e3b-4a46-8984-add1105b3db0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:38.242832 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24dd056-6e3b-4a46-8984-add1105b3db0 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:38.246422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f203f-5fd2-455d-94d9-639be16c9f6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:38.247026 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f203f-5fd2-455d-94d9-639be16c9f6b service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:38.253869 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f203f-5fd2-455d-94d9-639be16c9f6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:38.253869 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f203f-5fd2-455d-94d9-639be16c9f6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:38.256587 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4f203f-5fd2-455d-94d9-639be16c9f6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:38.258678 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4f203f-5fd2-455d-94d9-639be16c9f6b service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:38.268362 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1174/2331] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:07:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:07:38.676168 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7299b330-93c4-4ef9-a691-e6712a0d06a5 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:38.677624 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-7299b330-93c4-4ef9-a691-e6712a0d06a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:38.695253 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7299b330-93c4-4ef9-a691-e6712a0d06a5 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:38.700483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f19c84b-06d9-431a-a18f-9adaf354b689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:38.701174 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19c84b-06d9-431a-a18f-9adaf354b689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:38.701458 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19c84b-06d9-431a-a18f-9adaf354b689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:38.701730 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19c84b-06d9-431a-a18f-9adaf354b689 admin admin] PATH_INFO: `/v3/projects/287816ca60e74940a57777459509bf2e` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:38.703148 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f19c84b-06d9-431a-a18f-9adaf354b689 admin admin] RBAC: Authorizing `identity:delete_project(project_id=287816ca60e74940a57777459509bf2e)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:38.709685 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f19c84b-06d9-431a-a18f-9adaf354b689 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:38.810436 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2f19c84b-06d9-431a-a18f-9adaf354b689 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '287816ca60e74940a57777459509bf2e', 'request_id': 'req-2f19c84b-06d9-431a-a18f-9adaf354b689'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:38.891519 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2f19c84b-06d9-431a-a18f-9adaf354b689 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '287816ca60e74940a57777459509bf2e', 'request_id': 'req-2f19c84b-06d9-431a-a18f-9adaf354b689'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:38.899515 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2f19c84b-06d9-431a-a18f-9adaf354b689 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '287816ca60e74940a57777459509bf2e', 'request_id': 'req-2f19c84b-06d9-431a-a18f-9adaf354b689'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:38.907649 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:38.910757 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2f19c84b-06d9-431a-a18f-9adaf354b689 admin admin] The token cache is being invalidate because project 287816ca60e74940a57777459509bf2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:07:38.911054 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2f19c84b-06d9-431a-a18f-9adaf354b689 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:38.911709 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2f19c84b-06d9-431a-a18f-9adaf354b689 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:38.913802 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1158/2332] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:07:38 2022] DELETE /identity/v3/projects/287816ca60e74940a57777459509bf2e => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:07:38.918826 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62efb2-cbbd-4546-8a5b-eb22d4349723 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:38.919301 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62efb2-cbbd-4546-8a5b-eb22d4349723 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:38.919448 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62efb2-cbbd-4546-8a5b-eb22d4349723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:38.939483 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5a62efb2-cbbd-4546-8a5b-eb22d4349723 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:38.940724 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5a62efb2-cbbd-4546-8a5b-eb22d4349723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:39.020091 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1175/2333] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:07:39.029467 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259192b-3194-40a4-90ef-6633badf5ede None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:39.029752 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259192b-3194-40a4-90ef-6633badf5ede None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:39.030027 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259192b-3194-40a4-90ef-6633badf5ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:39.052116 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-b259192b-3194-40a4-90ef-6633badf5ede None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:39.053515 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b259192b-3194-40a4-90ef-6633badf5ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:39.061619 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1159/2334] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:07:39.124443 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62efb2-cbbd-4546-8a5b-eb22d4349723 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:39.134901 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62efb2-cbbd-4546-8a5b-eb22d4349723 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:39.139864 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee39e2ed-7071-47bc-b061-dcf20dca2295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:39.140502 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39e2ed-7071-47bc-b061-dcf20dca2295 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:39.140790 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39e2ed-7071-47bc-b061-dcf20dca2295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:39.141064 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39e2ed-7071-47bc-b061-dcf20dca2295 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:39.141721 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee39e2ed-7071-47bc-b061-dcf20dca2295 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:07:39.142029 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee39e2ed-7071-47bc-b061-dcf20dca2295 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:39.143634 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee39e2ed-7071-47bc-b061-dcf20dca2295 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:39.160796 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1176/2335] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:07:39.165461 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259192b-3194-40a4-90ef-6633badf5ede None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:39.188313 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259192b-3194-40a4-90ef-6633badf5ede None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:39.191898 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721bc11c-d816-4470-9a7e-592cb8c7ae03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:39.192644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721bc11c-d816-4470-9a7e-592cb8c7ae03 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:39.192997 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721bc11c-d816-4470-9a7e-592cb8c7ae03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:39.193340 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721bc11c-d816-4470-9a7e-592cb8c7ae03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:39.194047 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721bc11c-d816-4470-9a7e-592cb8c7ae03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:39.196599 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721bc11c-d816-4470-9a7e-592cb8c7ae03 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:39.333462 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1160/2336] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:07:39 2022] POST /identity/v3/projects => generated 383 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:07:39.338253 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee39e2ed-7071-47bc-b061-dcf20dca2295 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:39.348111 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee39e2ed-7071-47bc-b061-dcf20dca2295 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:39.408757 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9375b-a02d-4424-951a-2ee5062525b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:39.409364 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9375b-a02d-4424-951a-2ee5062525b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:39.409640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9375b-a02d-4424-951a-2ee5062525b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:39.409914 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9375b-a02d-4424-951a-2ee5062525b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:39.410547 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9375b-a02d-4424-951a-2ee5062525b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:39.412505 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9375b-a02d-4424-951a-2ee5062525b4 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:39.534513 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1177/2337] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:07:39 2022] POST /identity/v3/users => generated 292 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:07:39.536204 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721bc11c-d816-4470-9a7e-592cb8c7ae03 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:39.546592 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721bc11c-d816-4470-9a7e-592cb8c7ae03 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:39.552494 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e16f7-93e7-4e33-8972-6d9ed5082c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:39.554289 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e16f7-93e7-4e33-8972-6d9ed5082c7c admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:39.554687 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e16f7-93e7-4e33-8972-6d9ed5082c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:39.555160 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e16f7-93e7-4e33-8972-6d9ed5082c7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:39.555975 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e16f7-93e7-4e33-8972-6d9ed5082c7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:39.557955 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e16f7-93e7-4e33-8972-6d9ed5082c7c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:39.578634 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1161/2338] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:07:39.579921 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9375b-a02d-4424-951a-2ee5062525b4 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:39.591053 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9375b-a02d-4424-951a-2ee5062525b4 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:39.596390 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff8525-72ee-4c06-b500-c0315e3d2f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:39.597040 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff8525-72ee-4c06-b500-c0315e3d2f6b admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:39.597320 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff8525-72ee-4c06-b500-c0315e3d2f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:39.597596 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff8525-72ee-4c06-b500-c0315e3d2f6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:39.598244 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff8525-72ee-4c06-b500-c0315e3d2f6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:39.599946 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff8525-72ee-4c06-b500-c0315e3d2f6b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:39.605891 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:39.612154 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1178/2339] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:07:39.616764 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e16f7-93e7-4e33-8972-6d9ed5082c7c admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:39.626946 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e16f7-93e7-4e33-8972-6d9ed5082c7c admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:39.631565 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a00fe7-a003-4518-9456-5c61ef64a131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:39.632935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a00fe7-a003-4518-9456-5c61ef64a131 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:39.633241 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a00fe7-a003-4518-9456-5c61ef64a131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:39.633551 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a00fe7-a003-4518-9456-5c61ef64a131 admin admin] PATH_INFO: `/v3/projects/ce9d99ec2c254abaa341c4cab6f2c534/users/0d9129eaa2694ef6b3f006b67ef88c52/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:39.649943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a00fe7-a003-4518-9456-5c61ef64a131 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce9d99ec2c254abaa341c4cab6f2c534, user_id=0d9129eaa2694ef6b3f006b67ef88c52, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:39.652631 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a00fe7-a003-4518-9456-5c61ef64a131 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:39.721066 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1162/2340] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:07:39 2022] PUT /identity/v3/projects/ce9d99ec2c254abaa341c4cab6f2c534/users/0d9129eaa2694ef6b3f006b67ef88c52/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:07:39.725960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e337cf4-0e65-4450-be2a-4c72fb9b59b3 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:39.726349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e337cf4-0e65-4450-be2a-4c72fb9b59b3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:39.726684 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e337cf4-0e65-4450-be2a-4c72fb9b59b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:39.755806 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-4e337cf4-0e65-4450-be2a-4c72fb9b59b3 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:39.757690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4e337cf4-0e65-4450-be2a-4c72fb9b59b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:39.815398 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1179/2341] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:07:39.826255 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a00fe7-a003-4518-9456-5c61ef64a131 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:39.828306 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c2a00fe7-a003-4518-9456-5c61ef64a131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:39.890626 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a00fe7-a003-4518-9456-5c61ef64a131 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:39.894348 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24584da-812e-42c6-8085-3e7e4810cb3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:39.895030 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24584da-812e-42c6-8085-3e7e4810cb3c service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:39.895314 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24584da-812e-42c6-8085-3e7e4810cb3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:39.895587 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24584da-812e-42c6-8085-3e7e4810cb3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:39.904700 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24584da-812e-42c6-8085-3e7e4810cb3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:39.906128 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24584da-812e-42c6-8085-3e7e4810cb3c service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:39.916266 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1163/2342] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:07:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:07:41.343982 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2274b-0f13-4ad3-b9cc-d02e7f0b99e7 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:41.344529 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2274b-0f13-4ad3-b9cc-d02e7f0b99e7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:41.344690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2274b-0f13-4ad3-b9cc-d02e7f0b99e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:41.376769 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e3e2274b-0f13-4ad3-b9cc-d02e7f0b99e7 None None] MFA Rules not processed for user `0d9129eaa2694ef6b3f006b67ef88c52`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:41.377905 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e3e2274b-0f13-4ad3-b9cc-d02e7f0b99e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:41.452245 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1180/2343] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:07:41.465182 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24584da-812e-42c6-8085-3e7e4810cb3c service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:41.466597 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f24584da-812e-42c6-8085-3e7e4810cb3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:41.525482 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24584da-812e-42c6-8085-3e7e4810cb3c service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:41.529208 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2768aea-3790-437c-b438-a5703c3f0e08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:41.529819 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2768aea-3790-437c-b438-a5703c3f0e08 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:41.530092 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2768aea-3790-437c-b438-a5703c3f0e08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:41.530368 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2768aea-3790-437c-b438-a5703c3f0e08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:41.541312 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2768aea-3790-437c-b438-a5703c3f0e08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:41.542822 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2768aea-3790-437c-b438-a5703c3f0e08 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:41.551760 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1164/2344] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:07:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:07:44.464459 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db94f80-1d58-4185-b1c1-eeef859eee4d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:44.464943 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db94f80-1d58-4185-b1c1-eeef859eee4d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:44.465066 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db94f80-1d58-4185-b1c1-eeef859eee4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:44.484248 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9db94f80-1d58-4185-b1c1-eeef859eee4d None None] MFA Rules not processed for user `0d9129eaa2694ef6b3f006b67ef88c52`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:44.485454 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9db94f80-1d58-4185-b1c1-eeef859eee4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:44.495025 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1181/2345] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:07:44.794716 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf6f0c-348f-41ec-b872-525ff4ec07a4 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:44.795267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf6f0c-348f-41ec-b872-525ff4ec07a4 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:44.795267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf6f0c-348f-41ec-b872-525ff4ec07a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:44.815660 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-38cf6f0c-348f-41ec-b872-525ff4ec07a4 None None] MFA Rules not processed for user `c764d9778ef74b709cedbccabac37af0`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:44.818251 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-38cf6f0c-348f-41ec-b872-525ff4ec07a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:44.858460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:44.884254 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1165/2346] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:44 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:07:53.028000 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db94f80-1d58-4185-b1c1-eeef859eee4d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:53.033653 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9db94f80-1d58-4185-b1c1-eeef859eee4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:53.041944 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db94f80-1d58-4185-b1c1-eeef859eee4d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:53.045569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8209909-b745-4de9-8062-68821634ed13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:53.046178 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8209909-b745-4de9-8062-68821634ed13 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:53.046460 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8209909-b745-4de9-8062-68821634ed13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:53.046732 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8209909-b745-4de9-8062-68821634ed13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:53.048515 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c8209909-b745-4de9-8062-68821634ed13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:53.115404 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8209909-b745-4de9-8062-68821634ed13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:53.117544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8209909-b745-4de9-8062-68821634ed13 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:53.127673 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1182/2347] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:07:53 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:07:55.106777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf6f0c-348f-41ec-b872-525ff4ec07a4 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:55.112898 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-38cf6f0c-348f-41ec-b872-525ff4ec07a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:55.127220 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf6f0c-348f-41ec-b872-525ff4ec07a4 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:55.134046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66b505-be90-48dc-890e-5ec807118c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:55.141780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66b505-be90-48dc-890e-5ec807118c03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:55.141780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66b505-be90-48dc-890e-5ec807118c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:55.141780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66b505-be90-48dc-890e-5ec807118c03 admin admin] PATH_INFO: `/v3/users/8b179d722d9e4dac980a3c7b88c5fa0e` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:55.141780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f66b505-be90-48dc-890e-5ec807118c03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b179d722d9e4dac980a3c7b88c5fa0e)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:55.141780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f66b505-be90-48dc-890e-5ec807118c03 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:55.330995 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2f66b505-be90-48dc-890e-5ec807118c03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b179d722d9e4dac980a3c7b88c5fa0e'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:55.331658 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2f66b505-be90-48dc-890e-5ec807118c03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b179d722d9e4dac980a3c7b88c5fa0e'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:55.338767 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2f66b505-be90-48dc-890e-5ec807118c03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b179d722d9e4dac980a3c7b88c5fa0e'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:55.351221 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2f66b505-be90-48dc-890e-5ec807118c03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b179d722d9e4dac980a3c7b88c5fa0e'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:55.351902 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2f66b505-be90-48dc-890e-5ec807118c03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b179d722d9e4dac980a3c7b88c5fa0e'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:55.423592 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1166/2348] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:07:55 2022] DELETE /identity/v3/users/8b179d722d9e4dac980a3c7b88c5fa0e => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:07:55.427490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1af93-b078-4245-bc68-6bb6e0b7ebc7 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:55.428021 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1af93-b078-4245-bc68-6bb6e0b7ebc7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:55.428584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1af93-b078-4245-bc68-6bb6e0b7ebc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:55.451740 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b8a1af93-b078-4245-bc68-6bb6e0b7ebc7 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:55.453375 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b8a1af93-b078-4245-bc68-6bb6e0b7ebc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:55.466187 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:55.515556 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1183/2349] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:07:55.525875 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66b505-be90-48dc-890e-5ec807118c03 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:55.527546 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2f66b505-be90-48dc-890e-5ec807118c03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:55.586912 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66b505-be90-48dc-890e-5ec807118c03 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:55.590530 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b0021-7212-48a1-a5b5-0e597d06c150 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:55.591135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b0021-7212-48a1-a5b5-0e597d06c150 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:55.591414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b0021-7212-48a1-a5b5-0e597d06c150 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:55.591686 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b0021-7212-48a1-a5b5-0e597d06c150 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:55.602229 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b0021-7212-48a1-a5b5-0e597d06c150 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:55.603681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b0021-7212-48a1-a5b5-0e597d06c150 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:55.614495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1167/2350] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:07:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:07:56.078360 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a1af93-b078-4245-bc68-6bb6e0b7ebc7 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:56.080085 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b8a1af93-b078-4245-bc68-6bb6e0b7ebc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:56.094098 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a1af93-b078-4245-bc68-6bb6e0b7ebc7 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:56.099398 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00760329-3cee-40f8-99ba-3f2ce9773e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:56.100030 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00760329-3cee-40f8-99ba-3f2ce9773e42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:56.100530 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00760329-3cee-40f8-99ba-3f2ce9773e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:56.101083 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00760329-3cee-40f8-99ba-3f2ce9773e42 admin admin] PATH_INFO: `/v3/projects/a5ead2e7d5e5497bbe48196ef6d1acad` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:56.102215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00760329-3cee-40f8-99ba-3f2ce9773e42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5ead2e7d5e5497bbe48196ef6d1acad)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:56.104167 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00760329-3cee-40f8-99ba-3f2ce9773e42 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:56.262920 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-00760329-3cee-40f8-99ba-3f2ce9773e42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5ead2e7d5e5497bbe48196ef6d1acad', 'request_id': 'req-00760329-3cee-40f8-99ba-3f2ce9773e42'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:56.271413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-00760329-3cee-40f8-99ba-3f2ce9773e42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5ead2e7d5e5497bbe48196ef6d1acad', 'request_id': 'req-00760329-3cee-40f8-99ba-3f2ce9773e42'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:56.365690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-00760329-3cee-40f8-99ba-3f2ce9773e42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5ead2e7d5e5497bbe48196ef6d1acad', 'request_id': 'req-00760329-3cee-40f8-99ba-3f2ce9773e42'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:56.377756 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-00760329-3cee-40f8-99ba-3f2ce9773e42 admin admin] The token cache is being invalidate because project a5ead2e7d5e5497bbe48196ef6d1acad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:07:56.378221 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-00760329-3cee-40f8-99ba-3f2ce9773e42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:56.378765 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-00760329-3cee-40f8-99ba-3f2ce9773e42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:56.380637 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1184/2351] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:07:56 2022] DELETE /identity/v3/projects/a5ead2e7d5e5497bbe48196ef6d1acad => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:07:56.385206 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b0021-7212-48a1-a5b5-0e597d06c150 service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:56.396392 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b0021-7212-48a1-a5b5-0e597d06c150 service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:56.452940 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:56.474241 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2565629-4906-471c-a061-55041d802078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:56.474241 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2565629-4906-471c-a061-55041d802078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:56.474241 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2565629-4906-471c-a061-55041d802078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:56.480399 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2565629-4906-471c-a061-55041d802078 admin admin] PATH_INFO: `/v3/users/6553fa301c954f12b0e1e31b32307371` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:56.481947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2565629-4906-471c-a061-55041d802078 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6553fa301c954f12b0e1e31b32307371)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:56.484233 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2565629-4906-471c-a061-55041d802078 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:56.654587 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f2565629-4906-471c-a061-55041d802078 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6553fa301c954f12b0e1e31b32307371'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:56.655313 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f2565629-4906-471c-a061-55041d802078 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6553fa301c954f12b0e1e31b32307371'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:56.662739 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f2565629-4906-471c-a061-55041d802078 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6553fa301c954f12b0e1e31b32307371'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:56.683573 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f2565629-4906-471c-a061-55041d802078 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6553fa301c954f12b0e1e31b32307371'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:56.684707 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f2565629-4906-471c-a061-55041d802078 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6553fa301c954f12b0e1e31b32307371'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:56.749373 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1168/2352] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:07:56 2022] DELETE /identity/v3/users/6553fa301c954f12b0e1e31b32307371 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:07:57.528770 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00760329-3cee-40f8-99ba-3f2ce9773e42 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:57.530235 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-00760329-3cee-40f8-99ba-3f2ce9773e42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:57.546457 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00760329-3cee-40f8-99ba-3f2ce9773e42 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:57.635518 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105ae5c-8d6e-4fc8-8221-2bf8a58ea361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:57.636190 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105ae5c-8d6e-4fc8-8221-2bf8a58ea361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:57.636467 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105ae5c-8d6e-4fc8-8221-2bf8a58ea361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:57.636737 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105ae5c-8d6e-4fc8-8221-2bf8a58ea361 admin admin] PATH_INFO: `/v3/users/3a9591d38f5146849ae15fdd5685c445` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:57.637864 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4105ae5c-8d6e-4fc8-8221-2bf8a58ea361 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a9591d38f5146849ae15fdd5685c445)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:57.639794 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4105ae5c-8d6e-4fc8-8221-2bf8a58ea361 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:57.687674 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2565629-4906-471c-a061-55041d802078 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:57.689243 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f2565629-4906-471c-a061-55041d802078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:57.715420 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2565629-4906-471c-a061-55041d802078 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:57.720642 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43010e9e-233e-451d-9810-bd06b617a625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:57.721411 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43010e9e-233e-451d-9810-bd06b617a625 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:57.721961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43010e9e-233e-451d-9810-bd06b617a625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:57.722289 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43010e9e-233e-451d-9810-bd06b617a625 admin admin] PATH_INFO: `/v3/projects/ad437d1de20940c6b3f0f94a64ce3074` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:57.723624 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43010e9e-233e-451d-9810-bd06b617a625 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad437d1de20940c6b3f0f94a64ce3074)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:57.725522 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43010e9e-233e-451d-9810-bd06b617a625 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:57.900795 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:57.905899 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4105ae5c-8d6e-4fc8-8221-2bf8a58ea361 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a9591d38f5146849ae15fdd5685c445'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:57.907466 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4105ae5c-8d6e-4fc8-8221-2bf8a58ea361 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a9591d38f5146849ae15fdd5685c445'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:57.907948 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-43010e9e-233e-451d-9810-bd06b617a625 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad437d1de20940c6b3f0f94a64ce3074', 'request_id': 'req-43010e9e-233e-451d-9810-bd06b617a625'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:57.908210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4105ae5c-8d6e-4fc8-8221-2bf8a58ea361 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a9591d38f5146849ae15fdd5685c445'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:57.916234 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4105ae5c-8d6e-4fc8-8221-2bf8a58ea361 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a9591d38f5146849ae15fdd5685c445'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:57.929004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4105ae5c-8d6e-4fc8-8221-2bf8a58ea361 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a9591d38f5146849ae15fdd5685c445'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:57.999964 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-43010e9e-233e-451d-9810-bd06b617a625 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad437d1de20940c6b3f0f94a64ce3074', 'request_id': 'req-43010e9e-233e-451d-9810-bd06b617a625'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:58.007924 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-43010e9e-233e-451d-9810-bd06b617a625 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad437d1de20940c6b3f0f94a64ce3074', 'request_id': 'req-43010e9e-233e-451d-9810-bd06b617a625'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:58.019426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-43010e9e-233e-451d-9810-bd06b617a625 admin admin] The token cache is being invalidate because project ad437d1de20940c6b3f0f94a64ce3074 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:07:58.019718 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-43010e9e-233e-451d-9810-bd06b617a625 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:58.020391 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-43010e9e-233e-451d-9810-bd06b617a625 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:58.021966 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1169/2353] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:07:57 2022] DELETE /identity/v3/projects/ad437d1de20940c6b3f0f94a64ce3074 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:07:58.027088 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fb1b4-5f1e-4768-bce4-10118775078f None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:58.027370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fb1b4-5f1e-4768-bce4-10118775078f None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:58.027645 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fb1b4-5f1e-4768-bce4-10118775078f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:58.046908 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-543fb1b4-5f1e-4768-bce4-10118775078f None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:58.048163 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-543fb1b4-5f1e-4768-bce4-10118775078f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:58.082287 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1185/2354] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:07:57 2022] DELETE /identity/v3/users/3a9591d38f5146849ae15fdd5685c445 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:07:58.084329 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41e9bf-3bc6-41f4-babd-3de95a78f5ee None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:58.084736 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41e9bf-3bc6-41f4-babd-3de95a78f5ee None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:58.085702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41e9bf-3bc6-41f4-babd-3de95a78f5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:58.110570 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1d41e9bf-3bc6-41f4-babd-3de95a78f5ee None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:58.112205 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1d41e9bf-3bc6-41f4-babd-3de95a78f5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:58.120237 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:58.140253 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1170/2355] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:07:58.146320 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820feb39-29b6-47cd-9dcd-fd9a9f9c72d1 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:58.146320 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820feb39-29b6-47cd-9dcd-fd9a9f9c72d1 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:58.146320 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820feb39-29b6-47cd-9dcd-fd9a9f9c72d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:58.168225 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-820feb39-29b6-47cd-9dcd-fd9a9f9c72d1 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:58.169515 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-820feb39-29b6-47cd-9dcd-fd9a9f9c72d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:58.179209 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1186/2356] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:07:58.189134 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41e9bf-3bc6-41f4-babd-3de95a78f5ee None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:58.190595 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1d41e9bf-3bc6-41f4-babd-3de95a78f5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:58.232326 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1171/2357] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:07:58.243983 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820feb39-29b6-47cd-9dcd-fd9a9f9c72d1 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:58.251915 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820feb39-29b6-47cd-9dcd-fd9a9f9c72d1 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:58.259247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41e9bf-3bc6-41f4-babd-3de95a78f5ee None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:58.261644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b2938-3078-44af-a8c3-ad7c4a333f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:58.262362 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b2938-3078-44af-a8c3-ad7c4a333f8b admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:58.262824 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b2938-3078-44af-a8c3-ad7c4a333f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:58.263122 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b2938-3078-44af-a8c3-ad7c4a333f8b admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:58.263393 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085d165-66c1-4afd-9434-6180f2241365 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:58.263970 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936b2938-3078-44af-a8c3-ad7c4a333f8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:07:58.264133 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085d165-66c1-4afd-9434-6180f2241365 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:58.264317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085d165-66c1-4afd-9434-6180f2241365 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:58.264440 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936b2938-3078-44af-a8c3-ad7c4a333f8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:58.264595 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085d165-66c1-4afd-9434-6180f2241365 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:58.266193 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936b2938-3078-44af-a8c3-ad7c4a333f8b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:58.275800 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085d165-66c1-4afd-9434-6180f2241365 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:58.277286 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085d165-66c1-4afd-9434-6180f2241365 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:58.279758 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1172/2358] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:07:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:07:58.283906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b2938-3078-44af-a8c3-ad7c4a333f8b admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:58.287432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1187/2359] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:07:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:07:58.295016 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b2938-3078-44af-a8c3-ad7c4a333f8b admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:58.299947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff2c892-b28e-4b3d-8bfa-592cb0d7abdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:58.301122 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff2c892-b28e-4b3d-8bfa-592cb0d7abdc admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:58.301122 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff2c892-b28e-4b3d-8bfa-592cb0d7abdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:58.301122 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff2c892-b28e-4b3d-8bfa-592cb0d7abdc admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:58.301768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff2c892-b28e-4b3d-8bfa-592cb0d7abdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:58.303672 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff2c892-b28e-4b3d-8bfa-592cb0d7abdc admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:58.392971 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1173/2360] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:07:58 2022] POST /identity/v3/projects => generated 369 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:07:58.396318 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085d165-66c1-4afd-9434-6180f2241365 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:58.406411 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085d165-66c1-4afd-9434-6180f2241365 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:58.471315 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9305d-a2ff-43e2-957e-6e631d2859e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:58.471927 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9305d-a2ff-43e2-957e-6e631d2859e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:58.472239 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9305d-a2ff-43e2-957e-6e631d2859e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:58.472508 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9305d-a2ff-43e2-957e-6e631d2859e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:58.473197 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d9305d-a2ff-43e2-957e-6e631d2859e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:58.475140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d9305d-a2ff-43e2-957e-6e631d2859e0 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:58.486639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:58.600030 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1188/2361] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:07:58 2022] POST /identity/v3/users => generated 285 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:07:58.605218 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff2c892-b28e-4b3d-8bfa-592cb0d7abdc admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:58.627680 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff2c892-b28e-4b3d-8bfa-592cb0d7abdc admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:58.632913 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3acf00-a6cf-412f-97f5-41611d107af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:58.633802 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3acf00-a6cf-412f-97f5-41611d107af3 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:58.634172 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3acf00-a6cf-412f-97f5-41611d107af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:58.635545 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3acf00-a6cf-412f-97f5-41611d107af3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:58.635545 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3acf00-a6cf-412f-97f5-41611d107af3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:58.637779 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3acf00-a6cf-412f-97f5-41611d107af3 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:58.657434 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1174/2362] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:07:58 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:07:58.659109 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9305d-a2ff-43e2-957e-6e631d2859e0 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:58.672468 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9305d-a2ff-43e2-957e-6e631d2859e0 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:58.687227 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2440cd0-befa-4502-b771-5ff9b88eb090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:58.687849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2440cd0-befa-4502-b771-5ff9b88eb090 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:58.688166 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2440cd0-befa-4502-b771-5ff9b88eb090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:58.688447 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2440cd0-befa-4502-b771-5ff9b88eb090 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:58.689105 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2440cd0-befa-4502-b771-5ff9b88eb090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:58.690792 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2440cd0-befa-4502-b771-5ff9b88eb090 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:58.705846 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1189/2363] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:07:58 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:07:58.707544 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3acf00-a6cf-412f-97f5-41611d107af3 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:58.724492 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3acf00-a6cf-412f-97f5-41611d107af3 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:58.730065 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51eb90-c0ad-4bfa-9674-d6a3b8574cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:58.732497 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51eb90-c0ad-4bfa-9674-d6a3b8574cfd admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:58.732497 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51eb90-c0ad-4bfa-9674-d6a3b8574cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:58.732497 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51eb90-c0ad-4bfa-9674-d6a3b8574cfd admin admin] PATH_INFO: `/v3/projects/fc8b3345f6c74c38a7f3185ffe69f68e/users/de2af7470f3549a0939777ea7dda1886/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:58.748994 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51eb90-c0ad-4bfa-9674-d6a3b8574cfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc8b3345f6c74c38a7f3185ffe69f68e, user_id=de2af7470f3549a0939777ea7dda1886, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:58.752786 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51eb90-c0ad-4bfa-9674-d6a3b8574cfd admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:58.798313 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2440cd0-befa-4502-b771-5ff9b88eb090 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:58.800304 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d2440cd0-befa-4502-b771-5ff9b88eb090 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:58.828943 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2440cd0-befa-4502-b771-5ff9b88eb090 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:58.840124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1175/2364] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:07:58 2022] PUT /identity/v3/projects/fc8b3345f6c74c38a7f3185ffe69f68e/users/de2af7470f3549a0939777ea7dda1886/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:07:58.843424 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccaeaec-2046-4e23-bc80-f445c4acafb8 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:58.844588 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccaeaec-2046-4e23-bc80-f445c4acafb8 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:58.845066 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccaeaec-2046-4e23-bc80-f445c4acafb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:58.885168 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-7ccaeaec-2046-4e23-bc80-f445c4acafb8 None None] MFA Rules not processed for user `de2af7470f3549a0939777ea7dda1886`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:58.886414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-7ccaeaec-2046-4e23-bc80-f445c4acafb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:58.899051 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c673a40-99f8-4942-bcd6-bf2cad04b234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:58.900098 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c673a40-99f8-4942-bcd6-bf2cad04b234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:58.900659 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c673a40-99f8-4942-bcd6-bf2cad04b234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:58.901114 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c673a40-99f8-4942-bcd6-bf2cad04b234 admin admin] PATH_INFO: `/v3/projects/da98d88709a243ccabdb79a4524b68a7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:58.902547 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c673a40-99f8-4942-bcd6-bf2cad04b234 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da98d88709a243ccabdb79a4524b68a7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:58.904818 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c673a40-99f8-4942-bcd6-bf2cad04b234 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:58.958051 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1176/2365] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:07:58.963052 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccaeaec-2046-4e23-bc80-f445c4acafb8 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:58.977207 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccaeaec-2046-4e23-bc80-f445c4acafb8 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:58.982100 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18854a12-e6d9-4a65-94ed-79ed14abc1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:58.982770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18854a12-e6d9-4a65-94ed-79ed14abc1ab admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:58.983056 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18854a12-e6d9-4a65-94ed-79ed14abc1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:58.983379 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18854a12-e6d9-4a65-94ed-79ed14abc1ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:58.986253 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18854a12-e6d9-4a65-94ed-79ed14abc1ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:58.988440 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18854a12-e6d9-4a65-94ed-79ed14abc1ab admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:59.005944 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:59.008560 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8c673a40-99f8-4942-bcd6-bf2cad04b234 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da98d88709a243ccabdb79a4524b68a7', 'request_id': 'req-8c673a40-99f8-4942-bcd6-bf2cad04b234'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:59.015508 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8c673a40-99f8-4942-bcd6-bf2cad04b234 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da98d88709a243ccabdb79a4524b68a7', 'request_id': 'req-8c673a40-99f8-4942-bcd6-bf2cad04b234'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:59.093178 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1177/2366] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:07:58 2022] POST /identity/v3/projects => generated 365 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:07:59.096972 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18854a12-e6d9-4a65-94ed-79ed14abc1ab admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:59.113352 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8c673a40-99f8-4942-bcd6-bf2cad04b234 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da98d88709a243ccabdb79a4524b68a7', 'request_id': 'req-8c673a40-99f8-4942-bcd6-bf2cad04b234'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:59.127604 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8c673a40-99f8-4942-bcd6-bf2cad04b234 admin admin] The token cache is being invalidate because project da98d88709a243ccabdb79a4524b68a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:07:59.128023 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8c673a40-99f8-4942-bcd6-bf2cad04b234 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:59.128797 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8c673a40-99f8-4942-bcd6-bf2cad04b234 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:07:59.130449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1190/2367] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:07:58 2022] DELETE /identity/v3/projects/da98d88709a243ccabdb79a4524b68a7 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:07:59.135947 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef0043-e47d-4b3c-893f-06c194593f8d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:59.136272 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef0043-e47d-4b3c-893f-06c194593f8d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:59.136551 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef0043-e47d-4b3c-893f-06c194593f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:59.156281 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e9ef0043-e47d-4b3c-893f-06c194593f8d None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:59.157543 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e9ef0043-e47d-4b3c-893f-06c194593f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:59.161892 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18854a12-e6d9-4a65-94ed-79ed14abc1ab admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:59.165896 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1191/2368] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:07:59.166721 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15239d22-bd7b-46e3-807d-808c4d6437a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:59.167327 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15239d22-bd7b-46e3-807d-808c4d6437a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:59.167655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15239d22-bd7b-46e3-807d-808c4d6437a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:59.167931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15239d22-bd7b-46e3-807d-808c4d6437a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:59.168599 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15239d22-bd7b-46e3-807d-808c4d6437a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:59.170627 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15239d22-bd7b-46e3-807d-808c4d6437a0 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:59.173936 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e0ac4-d424-46e5-95df-3941053c5cf8 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:59.174226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e0ac4-d424-46e5-95df-3941053c5cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:59.174500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e0ac4-d424-46e5-95df-3941053c5cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:59.194067 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-326e0ac4-d424-46e5-95df-3941053c5cf8 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:59.195917 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-326e0ac4-d424-46e5-95df-3941053c5cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:59.204172 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1192/2369] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:07:59.208214 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e0ac4-d424-46e5-95df-3941053c5cf8 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:59.217158 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e0ac4-d424-46e5-95df-3941053c5cf8 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:59.221746 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d397c9-4c37-4c15-8690-11e9d0dc3d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:59.222412 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d397c9-4c37-4c15-8690-11e9d0dc3d86 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:59.222580 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d397c9-4c37-4c15-8690-11e9d0dc3d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:59.222853 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d397c9-4c37-4c15-8690-11e9d0dc3d86 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:59.223481 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d397c9-4c37-4c15-8690-11e9d0dc3d86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:07:59.223784 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d397c9-4c37-4c15-8690-11e9d0dc3d86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:59.225503 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d397c9-4c37-4c15-8690-11e9d0dc3d86 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:59.237029 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1193/2370] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:07:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:07:59.241761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d397c9-4c37-4c15-8690-11e9d0dc3d86 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:59.251870 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d397c9-4c37-4c15-8690-11e9d0dc3d86 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:59.256628 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2669ebb5-9a33-4f8d-9906-fb3d4bbf7b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:59.258493 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2669ebb5-9a33-4f8d-9906-fb3d4bbf7b37 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:59.258807 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2669ebb5-9a33-4f8d-9906-fb3d4bbf7b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:59.259081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2669ebb5-9a33-4f8d-9906-fb3d4bbf7b37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:59.259711 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2669ebb5-9a33-4f8d-9906-fb3d4bbf7b37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:59.261770 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2669ebb5-9a33-4f8d-9906-fb3d4bbf7b37 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:59.289803 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1178/2371] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:07:59 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:07:59.290843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15239d22-bd7b-46e3-807d-808c4d6437a0 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:59.292473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-15239d22-bd7b-46e3-807d-808c4d6437a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:59.304629 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15239d22-bd7b-46e3-807d-808c4d6437a0 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:59.310647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7119a-5d83-492f-9463-61a6a33a46d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:59.311263 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7119a-5d83-492f-9463-61a6a33a46d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:59.311550 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7119a-5d83-492f-9463-61a6a33a46d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:59.311825 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7119a-5d83-492f-9463-61a6a33a46d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:59.312559 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7119a-5d83-492f-9463-61a6a33a46d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:59.314268 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7119a-5d83-492f-9463-61a6a33a46d5 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:59.327795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1179/2372] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:07:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:07:59.329367 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7119a-5d83-492f-9463-61a6a33a46d5 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:59.340663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7119a-5d83-492f-9463-61a6a33a46d5 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:59.348275 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8900febd-73d9-4214-9f7b-be6b65f1c404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:59.349239 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900febd-73d9-4214-9f7b-be6b65f1c404 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:59.349595 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900febd-73d9-4214-9f7b-be6b65f1c404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:59.349939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900febd-73d9-4214-9f7b-be6b65f1c404 admin admin] PATH_INFO: `/v3/projects/60b7397460c04f44b9fd973dbbe367f0/users/3f2b01c0e97e42f4988849b891d2b62f/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:59.350229 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1194/2373] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:07:59 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:07:59.357487 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2669ebb5-9a33-4f8d-9906-fb3d4bbf7b37 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:59.374261 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8900febd-73d9-4214-9f7b-be6b65f1c404 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b7397460c04f44b9fd973dbbe367f0, user_id=3f2b01c0e97e42f4988849b891d2b62f, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:59.375780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8900febd-73d9-4214-9f7b-be6b65f1c404 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:59.420578 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2669ebb5-9a33-4f8d-9906-fb3d4bbf7b37 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:59.425313 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8136c5-8d4f-4519-b4c9-ebdab15e8281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:59.426101 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8136c5-8d4f-4519-b4c9-ebdab15e8281 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:59.426379 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8136c5-8d4f-4519-b4c9-ebdab15e8281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:59.426650 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8136c5-8d4f-4519-b4c9-ebdab15e8281 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:59.427282 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8136c5-8d4f-4519-b4c9-ebdab15e8281 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:59.429832 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8136c5-8d4f-4519-b4c9-ebdab15e8281 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:59.446449 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1180/2374] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:07:59 2022] PUT /identity/v3/projects/60b7397460c04f44b9fd973dbbe367f0/users/3f2b01c0e97e42f4988849b891d2b62f/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:07:59.452753 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8900febd-73d9-4214-9f7b-be6b65f1c404 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:59.513845 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:59.519408 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8900febd-73d9-4214-9f7b-be6b65f1c404 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:59.524193 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d0942-d6b3-4c81-b8f4-092f74cf8037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:59.524801 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d0942-d6b3-4c81-b8f4-092f74cf8037 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:59.525085 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d0942-d6b3-4c81-b8f4-092f74cf8037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:59.525494 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d0942-d6b3-4c81-b8f4-092f74cf8037 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:59.526144 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473d0942-d6b3-4c81-b8f4-092f74cf8037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:59.527749 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473d0942-d6b3-4c81-b8f4-092f74cf8037 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:59.537687 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1181/2375] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:07:59 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:07:59.542237 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d0942-d6b3-4c81-b8f4-092f74cf8037 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:59.548455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1195/2376] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:07:59 2022] POST /identity/v3/users => generated 288 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:07:59.552416 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8136c5-8d4f-4519-b4c9-ebdab15e8281 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:59.553178 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d0942-d6b3-4c81-b8f4-092f74cf8037 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:59.558696 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef4278-979a-4759-aa5f-0648200c596e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:59.559537 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef4278-979a-4759-aa5f-0648200c596e admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:59.559817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef4278-979a-4759-aa5f-0648200c596e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:59.560214 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef4278-979a-4759-aa5f-0648200c596e admin admin] PATH_INFO: `/v3/projects/60b7397460c04f44b9fd973dbbe367f0/users/3f2b01c0e97e42f4988849b891d2b62f/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:59.562083 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef4278-979a-4759-aa5f-0648200c596e admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b7397460c04f44b9fd973dbbe367f0, user_id=3f2b01c0e97e42f4988849b891d2b62f, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:59.562719 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8136c5-8d4f-4519-b4c9-ebdab15e8281 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:59.564871 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef4278-979a-4759-aa5f-0648200c596e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:59.567441 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476ba36-cac3-488b-9de8-440f4889836b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:59.568030 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476ba36-cac3-488b-9de8-440f4889836b admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:59.578455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476ba36-cac3-488b-9de8-440f4889836b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:59.578455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476ba36-cac3-488b-9de8-440f4889836b admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:59.578455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f476ba36-cac3-488b-9de8-440f4889836b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:59.578455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f476ba36-cac3-488b-9de8-440f4889836b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:59.578828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1182/2377] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:07:59 2022] PUT /identity/v3/projects/60b7397460c04f44b9fd973dbbe367f0/users/3f2b01c0e97e42f4988849b891d2b62f/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:07:59.582566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1196/2378] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:07:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:07:59.586896 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef4278-979a-4759-aa5f-0648200c596e admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:59.588173 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b786d9c-811b-43a6-a859-a713ea2e2236 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:59.588462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b786d9c-811b-43a6-a859-a713ea2e2236 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:59.588739 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b786d9c-811b-43a6-a859-a713ea2e2236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:59.616403 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9b786d9c-811b-43a6-a859-a713ea2e2236 None None] MFA Rules not processed for user `3f2b01c0e97e42f4988849b891d2b62f`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:59.617634 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9b786d9c-811b-43a6-a859-a713ea2e2236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:59.624383 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:07:59.652440 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef4278-979a-4759-aa5f-0648200c596e admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:59.657040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda5f7d-bdb7-425c-b706-eea728601add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:59.657633 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda5f7d-bdb7-425c-b706-eea728601add admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:59.657908 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda5f7d-bdb7-425c-b706-eea728601add admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:59.658176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda5f7d-bdb7-425c-b706-eea728601add admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:59.658810 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda5f7d-bdb7-425c-b706-eea728601add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:59.660440 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda5f7d-bdb7-425c-b706-eea728601add admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:59.670296 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1183/2379] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:07:59 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:07:59.674430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda5f7d-bdb7-425c-b706-eea728601add admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:59.684106 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda5f7d-bdb7-425c-b706-eea728601add admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:59.686511 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1197/2380] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:59 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:07:59.688769 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc541e3a-52f0-484d-8d58-8d16ae631a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:59.689438 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc541e3a-52f0-484d-8d58-8d16ae631a1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:59.689753 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc541e3a-52f0-484d-8d58-8d16ae631a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:59.690032 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc541e3a-52f0-484d-8d58-8d16ae631a1d admin admin] PATH_INFO: `/v3/projects/c8403c10131d45b7a3f34ae872ac9e9e/users/f289a1bdd79b4ba0bf9499edb7729c9a/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:59.699330 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b786d9c-811b-43a6-a859-a713ea2e2236 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:59.700703 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9b786d9c-811b-43a6-a859-a713ea2e2236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:59.706688 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc541e3a-52f0-484d-8d58-8d16ae631a1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8403c10131d45b7a3f34ae872ac9e9e, user_id=f289a1bdd79b4ba0bf9499edb7729c9a, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:59.709392 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc541e3a-52f0-484d-8d58-8d16ae631a1d admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:59.758799 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b786d9c-811b-43a6-a859-a713ea2e2236 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:07:59.762488 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532baefa-238b-4a75-b35b-be1ca2bb6e90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:07:59.763080 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532baefa-238b-4a75-b35b-be1ca2bb6e90 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:59.763355 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532baefa-238b-4a75-b35b-be1ca2bb6e90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:59.763623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532baefa-238b-4a75-b35b-be1ca2bb6e90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:59.772953 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532baefa-238b-4a75-b35b-be1ca2bb6e90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:07:59.774342 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532baefa-238b-4a75-b35b-be1ca2bb6e90 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:07:59.784575 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1198/2381] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:07:59 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:07:59.828314 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1184/2382] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:07:59 2022] PUT /identity/v3/projects/c8403c10131d45b7a3f34ae872ac9e9e/users/f289a1bdd79b4ba0bf9499edb7729c9a/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:07:59.833255 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341f679-417a-413e-aaf0-7d55a93cc449 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:07:59.833533 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341f679-417a-413e-aaf0-7d55a93cc449 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:07:59.833801 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341f679-417a-413e-aaf0-7d55a93cc449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:07:59.853298 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-4341f679-417a-413e-aaf0-7d55a93cc449 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:07:59.854499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4341f679-417a-413e-aaf0-7d55a93cc449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:59.925337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1199/2383] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:07:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:07:59.936425 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc541e3a-52f0-484d-8d58-8d16ae631a1d admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:07:59.940309 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-dc541e3a-52f0-484d-8d58-8d16ae631a1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:07:59.962553 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:08:00.010617 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc541e3a-52f0-484d-8d58-8d16ae631a1d admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:08:00.014911 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5412575-ad95-405d-9b28-2b5197511999 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:08:00.015869 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5412575-ad95-405d-9b28-2b5197511999 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:00.015869 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5412575-ad95-405d-9b28-2b5197511999 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:00.016156 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5412575-ad95-405d-9b28-2b5197511999 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:00.029775 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5412575-ad95-405d-9b28-2b5197511999 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:08:00.029775 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5412575-ad95-405d-9b28-2b5197511999 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:08:00.044247 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1185/2384] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:07:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:08:03.013736 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341f679-417a-413e-aaf0-7d55a93cc449 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:08:03.026865 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341f679-417a-413e-aaf0-7d55a93cc449 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:08:03.031972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379693e3-16f3-45dc-935a-2eff4d1963aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:08:03.032986 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379693e3-16f3-45dc-935a-2eff4d1963aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:03.034071 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379693e3-16f3-45dc-935a-2eff4d1963aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:03.034071 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379693e3-16f3-45dc-935a-2eff4d1963aa admin admin] PATH_INFO: `/v3/users/de2af7470f3549a0939777ea7dda1886` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:03.034741 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379693e3-16f3-45dc-935a-2eff4d1963aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=de2af7470f3549a0939777ea7dda1886)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:08:03.036759 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379693e3-16f3-45dc-935a-2eff4d1963aa admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:08:03.234663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-379693e3-16f3-45dc-935a-2eff4d1963aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de2af7470f3549a0939777ea7dda1886'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:08:03.235526 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-379693e3-16f3-45dc-935a-2eff4d1963aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de2af7470f3549a0939777ea7dda1886'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:08:03.236643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-379693e3-16f3-45dc-935a-2eff4d1963aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de2af7470f3549a0939777ea7dda1886'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:08:03.244404 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-379693e3-16f3-45dc-935a-2eff4d1963aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de2af7470f3549a0939777ea7dda1886'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:08:03.270610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-379693e3-16f3-45dc-935a-2eff4d1963aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de2af7470f3549a0939777ea7dda1886'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:08:03.360806 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1200/2385] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:08:03 2022] DELETE /identity/v3/users/de2af7470f3549a0939777ea7dda1886 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:08:03.366041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe17eb7-c6de-4426-bc93-e7271edfa693 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:03.367542 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe17eb7-c6de-4426-bc93-e7271edfa693 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:03.368263 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe17eb7-c6de-4426-bc93-e7271edfa693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:03.393835 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-cbe17eb7-c6de-4426-bc93-e7271edfa693 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:08:03.396663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-cbe17eb7-c6de-4426-bc93-e7271edfa693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:08:03.464659 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1186/2386] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:08:03.485452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379693e3-16f3-45dc-935a-2eff4d1963aa admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:08:03.485452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-379693e3-16f3-45dc-935a-2eff4d1963aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:08:03.552137 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379693e3-16f3-45dc-935a-2eff4d1963aa admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:08:03.556543 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f69306-a84b-448f-81d5-225c16dcc2e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:08:03.557328 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f69306-a84b-448f-81d5-225c16dcc2e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:03.557682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f69306-a84b-448f-81d5-225c16dcc2e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:03.558026 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f69306-a84b-448f-81d5-225c16dcc2e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:03.568710 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f69306-a84b-448f-81d5-225c16dcc2e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:08:03.570908 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f69306-a84b-448f-81d5-225c16dcc2e3 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:08:03.584697 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1201/2387] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:08:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:08:04.367709 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe17eb7-c6de-4426-bc93-e7271edfa693 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:08:04.370875 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-cbe17eb7-c6de-4426-bc93-e7271edfa693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:08:04.386686 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe17eb7-c6de-4426-bc93-e7271edfa693 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:08:04.393328 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab1b37-fdd9-45c9-87a2-4318edff8d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:08:04.400291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab1b37-fdd9-45c9-87a2-4318edff8d65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:04.400291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab1b37-fdd9-45c9-87a2-4318edff8d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:04.400291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab1b37-fdd9-45c9-87a2-4318edff8d65 admin admin] PATH_INFO: `/v3/projects/fc8b3345f6c74c38a7f3185ffe69f68e` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:04.400291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fab1b37-fdd9-45c9-87a2-4318edff8d65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc8b3345f6c74c38a7f3185ffe69f68e)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:08:04.400291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fab1b37-fdd9-45c9-87a2-4318edff8d65 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:08:04.523867 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:08:04.527922 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0fab1b37-fdd9-45c9-87a2-4318edff8d65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc8b3345f6c74c38a7f3185ffe69f68e', 'request_id': 'req-0fab1b37-fdd9-45c9-87a2-4318edff8d65'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:08:04.631443 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0fab1b37-fdd9-45c9-87a2-4318edff8d65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc8b3345f6c74c38a7f3185ffe69f68e', 'request_id': 'req-0fab1b37-fdd9-45c9-87a2-4318edff8d65'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:08:04.638308 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0fab1b37-fdd9-45c9-87a2-4318edff8d65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc8b3345f6c74c38a7f3185ffe69f68e', 'request_id': 'req-0fab1b37-fdd9-45c9-87a2-4318edff8d65'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:08:04.657790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0fab1b37-fdd9-45c9-87a2-4318edff8d65 admin admin] The token cache is being invalidate because project fc8b3345f6c74c38a7f3185ffe69f68e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:08:04.658838 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0fab1b37-fdd9-45c9-87a2-4318edff8d65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:08:04.659158 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0fab1b37-fdd9-45c9-87a2-4318edff8d65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:08:04.662156 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1187/2388] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:08:04 2022] DELETE /identity/v3/projects/fc8b3345f6c74c38a7f3185ffe69f68e => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:08:04.666032 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f69306-a84b-448f-81d5-225c16dcc2e3 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:08:04.678034 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f69306-a84b-448f-81d5-225c16dcc2e3 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:08:04.749937 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:08:04.770745 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7979329-eb40-486c-ad31-45a3783db464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:08:04.771400 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7979329-eb40-486c-ad31-45a3783db464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:04.771684 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7979329-eb40-486c-ad31-45a3783db464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:04.771958 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7979329-eb40-486c-ad31-45a3783db464 admin admin] PATH_INFO: `/v3/users/3f2b01c0e97e42f4988849b891d2b62f` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:04.773452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7979329-eb40-486c-ad31-45a3783db464 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f2b01c0e97e42f4988849b891d2b62f)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:08:04.775520 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7979329-eb40-486c-ad31-45a3783db464 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:08:04.977220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d7979329-eb40-486c-ad31-45a3783db464 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f2b01c0e97e42f4988849b891d2b62f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:08:04.978460 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d7979329-eb40-486c-ad31-45a3783db464 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f2b01c0e97e42f4988849b891d2b62f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:08:04.979124 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d7979329-eb40-486c-ad31-45a3783db464 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f2b01c0e97e42f4988849b891d2b62f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:08:04.993290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d7979329-eb40-486c-ad31-45a3783db464 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f2b01c0e97e42f4988849b891d2b62f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:08:05.012837 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d7979329-eb40-486c-ad31-45a3783db464 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f2b01c0e97e42f4988849b891d2b62f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:08:05.092152 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1202/2389] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:08:04 2022] DELETE /identity/v3/users/3f2b01c0e97e42f4988849b891d2b62f => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:08:05.643968 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab1b37-fdd9-45c9-87a2-4318edff8d65 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:08:05.646576 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0fab1b37-fdd9-45c9-87a2-4318edff8d65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:08:05.674505 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab1b37-fdd9-45c9-87a2-4318edff8d65 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:08:05.857634 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb927aa9-b65b-4306-bcd8-c7f9e1776508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:08:05.858360 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb927aa9-b65b-4306-bcd8-c7f9e1776508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:05.858656 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb927aa9-b65b-4306-bcd8-c7f9e1776508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:05.858931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb927aa9-b65b-4306-bcd8-c7f9e1776508 admin admin] PATH_INFO: `/v3/projects/60b7397460c04f44b9fd973dbbe367f0` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:05.865106 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb927aa9-b65b-4306-bcd8-c7f9e1776508 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60b7397460c04f44b9fd973dbbe367f0)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:08:05.867154 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb927aa9-b65b-4306-bcd8-c7f9e1776508 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:08:05.983029 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a77702-2329-4951-b6cb-ba4ebbef6ae6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:05.987447 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a77702-2329-4951-b6cb-ba4ebbef6ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:05.987973 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a77702-2329-4951-b6cb-ba4ebbef6ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:06.045656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-11a77702-2329-4951-b6cb-ba4ebbef6ae6 None None] MFA Rules not processed for user `f289a1bdd79b4ba0bf9499edb7729c9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:08:06.046920 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-11a77702-2329-4951-b6cb-ba4ebbef6ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:08:06.084580 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-eb927aa9-b65b-4306-bcd8-c7f9e1776508 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60b7397460c04f44b9fd973dbbe367f0', 'request_id': 'req-eb927aa9-b65b-4306-bcd8-c7f9e1776508'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:08:06.123336 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1203/2390] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:08:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:08:06.129285 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a77702-2329-4951-b6cb-ba4ebbef6ae6 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:08:06.130680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-11a77702-2329-4951-b6cb-ba4ebbef6ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:08:06.147142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a77702-2329-4951-b6cb-ba4ebbef6ae6 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:08:06.154488 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-eb927aa9-b65b-4306-bcd8-c7f9e1776508 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60b7397460c04f44b9fd973dbbe367f0', 'request_id': 'req-eb927aa9-b65b-4306-bcd8-c7f9e1776508'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:08:06.163704 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-eb927aa9-b65b-4306-bcd8-c7f9e1776508 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60b7397460c04f44b9fd973dbbe367f0', 'request_id': 'req-eb927aa9-b65b-4306-bcd8-c7f9e1776508'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:08:06.183168 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-eb927aa9-b65b-4306-bcd8-c7f9e1776508 admin admin] The token cache is being invalidate because project 60b7397460c04f44b9fd973dbbe367f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:08:06.183468 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-eb927aa9-b65b-4306-bcd8-c7f9e1776508 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:08:06.184185 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-eb927aa9-b65b-4306-bcd8-c7f9e1776508 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:08:06.186544 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1188/2391] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:08:05 2022] DELETE /identity/v3/projects/60b7397460c04f44b9fd973dbbe367f0 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:08:06.191391 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcaf315-a5a1-45bb-a336-065c07ebfa28 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:06.191676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcaf315-a5a1-45bb-a336-065c07ebfa28 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:06.191949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcaf315-a5a1-45bb-a336-065c07ebfa28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:06.215051 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-dfcaf315-a5a1-45bb-a336-065c07ebfa28 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:08:06.216486 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-dfcaf315-a5a1-45bb-a336-065c07ebfa28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:08:06.219334 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:08:06.227064 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c9ef2-7e52-45cd-9da7-7892be016401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:08:06.227684 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c9ef2-7e52-45cd-9da7-7892be016401 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:06.227961 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c9ef2-7e52-45cd-9da7-7892be016401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:06.228518 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c9ef2-7e52-45cd-9da7-7892be016401 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:06.229161 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2c9ef2-7e52-45cd-9da7-7892be016401 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:08:06.231432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2c9ef2-7e52-45cd-9da7-7892be016401 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:08:06.238779 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1189/2392] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:08:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:08:06.242510 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa372fca-ee5a-442b-8331-33fd98b6b831 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:06.242510 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa372fca-ee5a-442b-8331-33fd98b6b831 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:06.242510 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa372fca-ee5a-442b-8331-33fd98b6b831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:06.267142 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-fa372fca-ee5a-442b-8331-33fd98b6b831 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:08:06.268659 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-fa372fca-ee5a-442b-8331-33fd98b6b831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:08:06.277789 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1190/2393] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:08:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:08:06.282147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa372fca-ee5a-442b-8331-33fd98b6b831 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:08:06.293578 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa372fca-ee5a-442b-8331-33fd98b6b831 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:08:06.298836 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f72f24-9695-403c-a178-4d6eb79c3e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:08:06.299876 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f72f24-9695-403c-a178-4d6eb79c3e62 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:06.300358 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f72f24-9695-403c-a178-4d6eb79c3e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:06.300866 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f72f24-9695-403c-a178-4d6eb79c3e62 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:06.301567 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f72f24-9695-403c-a178-4d6eb79c3e62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:08:06.302449 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f72f24-9695-403c-a178-4d6eb79c3e62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:08:06.304734 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f72f24-9695-403c-a178-4d6eb79c3e62 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:08:06.320981 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1191/2394] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:08:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:08:06.324775 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f72f24-9695-403c-a178-4d6eb79c3e62 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:08:06.331935 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1204/2395] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:08:06 2022] POST /identity/v3/projects => generated 379 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:08:06.336455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c9ef2-7e52-45cd-9da7-7892be016401 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:08:06.339074 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0c2c9ef2-7e52-45cd-9da7-7892be016401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:08:06.339843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f72f24-9695-403c-a178-4d6eb79c3e62 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:08:06.381445 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:08:06.436716 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524da9f-ef08-4b75-aa52-ce299916a1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:08:06.438161 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524da9f-ef08-4b75-aa52-ce299916a1f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:06.438161 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524da9f-ef08-4b75-aa52-ce299916a1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:06.438161 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524da9f-ef08-4b75-aa52-ce299916a1f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:06.438716 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c524da9f-ef08-4b75-aa52-ce299916a1f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:08:06.440678 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c524da9f-ef08-4b75-aa52-ce299916a1f5 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:08:06.479899 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c9ef2-7e52-45cd-9da7-7892be016401 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:08:06.498083 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ffc0f-578e-4469-a6ec-e8271000a7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:08:06.498760 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ffc0f-578e-4469-a6ec-e8271000a7d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:06.501464 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ffc0f-578e-4469-a6ec-e8271000a7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:06.501738 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ffc0f-578e-4469-a6ec-e8271000a7d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:06.502400 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12ffc0f-578e-4469-a6ec-e8271000a7d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:08:06.509305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12ffc0f-578e-4469-a6ec-e8271000a7d5 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:08:06.540900 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1192/2396] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:08:06 2022] POST /identity/v3/projects => generated 375 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:08:06.544932 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524da9f-ef08-4b75-aa52-ce299916a1f5 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:08:06.633282 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1205/2397] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:08:06 2022] POST /identity/v3/users => generated 296 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:08:06.635702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ffc0f-578e-4469-a6ec-e8271000a7d5 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:08:06.659379 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524da9f-ef08-4b75-aa52-ce299916a1f5 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:08:06.664509 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ffc0f-578e-4469-a6ec-e8271000a7d5 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:08:06.671759 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef984387-b9a9-482c-9c1e-58a668b3b74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:08:06.672627 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef984387-b9a9-482c-9c1e-58a668b3b74b admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:06.673361 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef984387-b9a9-482c-9c1e-58a668b3b74b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:06.673678 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef984387-b9a9-482c-9c1e-58a668b3b74b admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:06.674607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef984387-b9a9-482c-9c1e-58a668b3b74b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:08:06.676773 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef984387-b9a9-482c-9c1e-58a668b3b74b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:08:06.687763 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a1456c-6d4f-491f-ac29-4532a018734e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:08:06.688437 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1456c-6d4f-491f-ac29-4532a018734e admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:06.688718 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1456c-6d4f-491f-ac29-4532a018734e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:06.689151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1456c-6d4f-491f-ac29-4532a018734e admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:06.690752 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a1456c-6d4f-491f-ac29-4532a018734e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:08:06.691507 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a1456c-6d4f-491f-ac29-4532a018734e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:08:06.704410 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1206/2398] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:08:06.707481 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a1456c-6d4f-491f-ac29-4532a018734e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:08:06.721132 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a1456c-6d4f-491f-ac29-4532a018734e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:08:06.728114 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f93f52-7af9-447d-9e35-ebc11b55bdbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:08:06.728990 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f93f52-7af9-447d-9e35-ebc11b55bdbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:06.729317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f93f52-7af9-447d-9e35-ebc11b55bdbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:06.729670 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f93f52-7af9-447d-9e35-ebc11b55bdbc admin admin] PATH_INFO: `/v3/projects/07aa76b3c415436f83339df773c696fb/users/3274028787bf4d9f9797906c9d1d9507/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:06.752749 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f93f52-7af9-447d-9e35-ebc11b55bdbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=07aa76b3c415436f83339df773c696fb, user_id=3274028787bf4d9f9797906c9d1d9507, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:08:06.758047 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f93f52-7af9-447d-9e35-ebc11b55bdbc admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:08:06.796039 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1193/2399] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:08:06 2022] POST /identity/v3/users => generated 288 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:08:06.800192 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef984387-b9a9-482c-9c1e-58a668b3b74b admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:08:06.824218 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef984387-b9a9-482c-9c1e-58a668b3b74b admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:08:06.824218 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a9672-0741-4b16-8067-ce63412d3d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:08:06.824218 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a9672-0741-4b16-8067-ce63412d3d41 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:06.824218 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a9672-0741-4b16-8067-ce63412d3d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:06.824628 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a9672-0741-4b16-8067-ce63412d3d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:06.824628 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3a9672-0741-4b16-8067-ce63412d3d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:08:06.824628 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3a9672-0741-4b16-8067-ce63412d3d41 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:08:06.838410 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1194/2400] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:08:06.843127 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a9672-0741-4b16-8067-ce63412d3d41 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:08:06.856327 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a9672-0741-4b16-8067-ce63412d3d41 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:08:06.862655 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1207/2401] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:08:06 2022] PUT /identity/v3/projects/07aa76b3c415436f83339df773c696fb/users/3274028787bf4d9f9797906c9d1d9507/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:08:06.866130 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f93f52-7af9-447d-9e35-ebc11b55bdbc admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:08:06.924346 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6618d99d-f08f-48cb-a4cd-8f429d9f6495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:08:06.924969 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6618d99d-f08f-48cb-a4cd-8f429d9f6495 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:06.925253 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6618d99d-f08f-48cb-a4cd-8f429d9f6495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:06.925524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6618d99d-f08f-48cb-a4cd-8f429d9f6495 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:06.926196 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6618d99d-f08f-48cb-a4cd-8f429d9f6495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:08:06.927790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6618d99d-f08f-48cb-a4cd-8f429d9f6495 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:08:06.944706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1195/2402] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:08:06.945742 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6618d99d-f08f-48cb-a4cd-8f429d9f6495 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:08:06.951674 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f93f52-7af9-447d-9e35-ebc11b55bdbc admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:08:06.957937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6618d99d-f08f-48cb-a4cd-8f429d9f6495 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:08:06.960864 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c646f5-3ee2-4f61-89dc-2343f21a18a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:08:06.961490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c646f5-3ee2-4f61-89dc-2343f21a18a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:06.961782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c646f5-3ee2-4f61-89dc-2343f21a18a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:06.962054 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c646f5-3ee2-4f61-89dc-2343f21a18a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:06.962715 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c646f5-3ee2-4f61-89dc-2343f21a18a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:08:06.964475 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c646f5-3ee2-4f61-89dc-2343f21a18a6 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:08:06.964722 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efc933-2ec9-4f37-9d4b-2bf86377102e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:08:06.965396 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efc933-2ec9-4f37-9d4b-2bf86377102e admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:06.965673 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efc933-2ec9-4f37-9d4b-2bf86377102e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:06.966189 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efc933-2ec9-4f37-9d4b-2bf86377102e admin admin] PATH_INFO: `/v3/projects/984a72f1d3564c31aaef8970b9f3c077/users/47d365c0801c405f9a7384ed2ca5bfb5/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:06.976334 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1208/2403] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:08:06.983454 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5efc933-2ec9-4f37-9d4b-2bf86377102e admin admin] RBAC: Authorizing `identity:create_grant(project_id=984a72f1d3564c31aaef8970b9f3c077, user_id=47d365c0801c405f9a7384ed2ca5bfb5, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:08:06.987485 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5efc933-2ec9-4f37-9d4b-2bf86377102e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:08:06.992316 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c646f5-3ee2-4f61-89dc-2343f21a18a6 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:08:07.003131 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c646f5-3ee2-4f61-89dc-2343f21a18a6 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:08:07.009128 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b8909-9367-4382-9068-3686296daeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:08:07.010078 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b8909-9367-4382-9068-3686296daeae admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:07.010431 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b8909-9367-4382-9068-3686296daeae admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:07.010712 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b8909-9367-4382-9068-3686296daeae admin admin] PATH_INFO: `/v3/projects/07aa76b3c415436f83339df773c696fb/users/3274028787bf4d9f9797906c9d1d9507/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:07.012678 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b8909-9367-4382-9068-3686296daeae admin admin] RBAC: Authorizing `identity:create_grant(project_id=07aa76b3c415436f83339df773c696fb, user_id=3274028787bf4d9f9797906c9d1d9507, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:08:07.015611 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b8909-9367-4382-9068-3686296daeae admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:08:07.027874 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1209/2404] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:08:06 2022] PUT /identity/v3/projects/07aa76b3c415436f83339df773c696fb/users/3274028787bf4d9f9797906c9d1d9507/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:08:07.063788 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1196/2405] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:08:06 2022] PUT /identity/v3/projects/984a72f1d3564c31aaef8970b9f3c077/users/47d365c0801c405f9a7384ed2ca5bfb5/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:08:07.074947 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68434c35-f1e8-48ec-b89f-2d889742cab2 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:07.075234 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68434c35-f1e8-48ec-b89f-2d889742cab2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:07.075510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68434c35-f1e8-48ec-b89f-2d889742cab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:07.107148 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-68434c35-f1e8-48ec-b89f-2d889742cab2 None None] MFA Rules not processed for user `47d365c0801c405f9a7384ed2ca5bfb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:08:07.108398 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-68434c35-f1e8-48ec-b89f-2d889742cab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:08:07.114141 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:08:07.194751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1210/2406] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:08:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:08:07.199344 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efc933-2ec9-4f37-9d4b-2bf86377102e admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:08:07.213580 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:08:07.263811 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efc933-2ec9-4f37-9d4b-2bf86377102e admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:08:07.268661 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf64ae68-6f87-4045-98a5-ebdc465a93f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:08:07.269413 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64ae68-6f87-4045-98a5-ebdc465a93f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:07.269698 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64ae68-6f87-4045-98a5-ebdc465a93f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:07.269983 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64ae68-6f87-4045-98a5-ebdc465a93f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:07.270608 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf64ae68-6f87-4045-98a5-ebdc465a93f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:08:07.272666 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf64ae68-6f87-4045-98a5-ebdc465a93f2 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:08:07.374788 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1197/2407] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:08:07 2022] POST /identity/v3/projects => generated 377 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:08:07.379018 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68434c35-f1e8-48ec-b89f-2d889742cab2 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:08:07.399625 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68434c35-f1e8-48ec-b89f-2d889742cab2 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:08:07.461613 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05fabb7-79c6-4fac-9ca4-b417749277ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:08:07.462238 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05fabb7-79c6-4fac-9ca4-b417749277ea admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:07.462517 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05fabb7-79c6-4fac-9ca4-b417749277ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:07.462788 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05fabb7-79c6-4fac-9ca4-b417749277ea admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:07.463433 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05fabb7-79c6-4fac-9ca4-b417749277ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:08:07.465414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05fabb7-79c6-4fac-9ca4-b417749277ea admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:08:07.566582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1211/2408] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:08:07 2022] POST /identity/v3/users => generated 295 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:08:07.571233 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf64ae68-6f87-4045-98a5-ebdc465a93f2 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:08:07.585871 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf64ae68-6f87-4045-98a5-ebdc465a93f2 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:08:07.591677 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0f481-427d-4634-a7b3-93789874cda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:08:07.592534 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0f481-427d-4634-a7b3-93789874cda7 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:07.592967 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0f481-427d-4634-a7b3-93789874cda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:07.593566 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0f481-427d-4634-a7b3-93789874cda7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:07.599484 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c0f481-427d-4634-a7b3-93789874cda7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:08:07.601779 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c0f481-427d-4634-a7b3-93789874cda7 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:08:07.620061 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1198/2409] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:08:07.625405 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05fabb7-79c6-4fac-9ca4-b417749277ea admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:08:07.647789 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05fabb7-79c6-4fac-9ca4-b417749277ea admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:08:07.653874 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e059dd-5fe8-472a-804b-3bbfd60f2305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:08:07.654805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e059dd-5fe8-472a-804b-3bbfd60f2305 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:07.655189 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e059dd-5fe8-472a-804b-3bbfd60f2305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:07.656253 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e059dd-5fe8-472a-804b-3bbfd60f2305 admin admin] PATH_INFO: `/v3/projects/aa7e936295b5471ca36999c614bed29c/users/ea5cceb96da644bdb2c4b4b77cc46635/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:07.674950 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e059dd-5fe8-472a-804b-3bbfd60f2305 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa7e936295b5471ca36999c614bed29c, user_id=ea5cceb96da644bdb2c4b4b77cc46635, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:08:07.677957 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e059dd-5fe8-472a-804b-3bbfd60f2305 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:08:07.761100 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1212/2410] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:08:07 2022] PUT /identity/v3/projects/aa7e936295b5471ca36999c614bed29c/users/ea5cceb96da644bdb2c4b4b77cc46635/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:08:07.765261 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0f481-427d-4634-a7b3-93789874cda7 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:08:07.775934 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0f481-427d-4634-a7b3-93789874cda7 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:08:07.841393 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1970df-7c6e-4612-abe1-d67bacaf3c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:08:07.842181 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1970df-7c6e-4612-abe1-d67bacaf3c35 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:07.842495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1970df-7c6e-4612-abe1-d67bacaf3c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:07.842830 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1970df-7c6e-4612-abe1-d67bacaf3c35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:07.843571 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1970df-7c6e-4612-abe1-d67bacaf3c35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:08:07.845518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1970df-7c6e-4612-abe1-d67bacaf3c35 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:08:07.857745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1199/2411] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:08:07.861708 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e059dd-5fe8-472a-804b-3bbfd60f2305 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:08:07.873878 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e059dd-5fe8-472a-804b-3bbfd60f2305 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:08:07.880983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8014d-9c86-485e-8dc0-de05bbcf46c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:08:07.881850 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8014d-9c86-485e-8dc0-de05bbcf46c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:07.882245 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8014d-9c86-485e-8dc0-de05bbcf46c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:07.882608 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8014d-9c86-485e-8dc0-de05bbcf46c9 admin admin] PATH_INFO: `/v3/projects/aa7e936295b5471ca36999c614bed29c/users/ea5cceb96da644bdb2c4b4b77cc46635/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:07.885223 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c8014d-9c86-485e-8dc0-de05bbcf46c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa7e936295b5471ca36999c614bed29c, user_id=ea5cceb96da644bdb2c4b4b77cc46635, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:08:07.889382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c8014d-9c86-485e-8dc0-de05bbcf46c9 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:08:07.901218 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1213/2412] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:08:07 2022] PUT /identity/v3/projects/aa7e936295b5471ca36999c614bed29c/users/ea5cceb96da644bdb2c4b4b77cc46635/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:08:07.912696 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f7b10-23a8-4bc2-bd01-069a9252dbee None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:07.913515 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f7b10-23a8-4bc2-bd01-069a9252dbee None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:07.914019 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f7b10-23a8-4bc2-bd01-069a9252dbee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:07.947299 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-e18f7b10-23a8-4bc2-bd01-069a9252dbee None None] MFA Rules not processed for user `ea5cceb96da644bdb2c4b4b77cc46635`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:08:07.949426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e18f7b10-23a8-4bc2-bd01-069a9252dbee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:08:08.047685 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1200/2413] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:08:07 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:08:08.060850 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8014d-9c86-485e-8dc0-de05bbcf46c9 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:08:08.063739 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e5c8014d-9c86-485e-8dc0-de05bbcf46c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:08:08.136425 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8014d-9c86-485e-8dc0-de05bbcf46c9 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:08:08.141225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63969a6-5e1a-4e0b-a1fa-40906ae5cd9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:08:08.142070 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63969a6-5e1a-4e0b-a1fa-40906ae5cd9b service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:08.142491 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63969a6-5e1a-4e0b-a1fa-40906ae5cd9b service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:08.142933 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63969a6-5e1a-4e0b-a1fa-40906ae5cd9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:08.162342 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:08:08.223558 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63969a6-5e1a-4e0b-a1fa-40906ae5cd9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:08:08.225380 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63969a6-5e1a-4e0b-a1fa-40906ae5cd9b service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:08:08.236135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1214/2414] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:08:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:08:13.271073 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85143e9b-efa9-49a3-b205-b0e632d4d351 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:13.271973 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85143e9b-efa9-49a3-b205-b0e632d4d351 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:13.271973 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85143e9b-efa9-49a3-b205-b0e632d4d351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:13.290199 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:08:13.306363 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-85143e9b-efa9-49a3-b205-b0e632d4d351 None None] MFA Rules not processed for user `3274028787bf4d9f9797906c9d1d9507`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:08:13.307586 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-85143e9b-efa9-49a3-b205-b0e632d4d351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:08:13.382587 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1201/2415] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:08:13 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:08:13.396349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63969a6-5e1a-4e0b-a1fa-40906ae5cd9b service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:08:13.398918 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d63969a6-5e1a-4e0b-a1fa-40906ae5cd9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:08:13.476624 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63969a6-5e1a-4e0b-a1fa-40906ae5cd9b service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:08:13.482352 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b004d9f3-296b-4429-ae80-b28339eca702 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:08:13.482968 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004d9f3-296b-4429-ae80-b28339eca702 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:13.483246 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004d9f3-296b-4429-ae80-b28339eca702 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:13.483518 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004d9f3-296b-4429-ae80-b28339eca702 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:13.485249 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b004d9f3-296b-4429-ae80-b28339eca702 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:08:13.544396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b004d9f3-296b-4429-ae80-b28339eca702 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:08:13.546014 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b004d9f3-296b-4429-ae80-b28339eca702 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:08:13.556725 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1215/2416] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:08:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:08:19.063576 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85143e9b-efa9-49a3-b205-b0e632d4d351 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:08:19.068231 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-85143e9b-efa9-49a3-b205-b0e632d4d351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:08:19.078947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85143e9b-efa9-49a3-b205-b0e632d4d351 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:08:19.083102 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8309ae51-0ef2-41de-8321-3a60982280f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:08:19.083791 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8309ae51-0ef2-41de-8321-3a60982280f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:08:19.084097 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8309ae51-0ef2-41de-8321-3a60982280f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:08:19.084389 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8309ae51-0ef2-41de-8321-3a60982280f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:08:19.094942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8309ae51-0ef2-41de-8321-3a60982280f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:08:19.096427 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8309ae51-0ef2-41de-8321-3a60982280f0 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:08:19.106613 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1202/2417] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:08:19 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:08:23.551274 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=77 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:08:29.102121 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=37 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:09:06.348324 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b004d9f3-296b-4429-ae80-b28339eca702 service nova] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:06.413575 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b004d9f3-296b-4429-ae80-b28339eca702 service nova] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:06.418151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d03c67-433f-4a02-b04c-175cceb37de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:06.419457 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d03c67-433f-4a02-b04c-175cceb37de6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:06.419806 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d03c67-433f-4a02-b04c-175cceb37de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:06.420175 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d03c67-433f-4a02-b04c-175cceb37de6 admin admin] PATH_INFO: `/v3/users/47d365c0801c405f9a7384ed2ca5bfb5` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:06.421376 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d03c67-433f-4a02-b04c-175cceb37de6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47d365c0801c405f9a7384ed2ca5bfb5)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:06.423360 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d03c67-433f-4a02-b04c-175cceb37de6 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:06.561959 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-60d03c67-433f-4a02-b04c-175cceb37de6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47d365c0801c405f9a7384ed2ca5bfb5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:06.562605 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-60d03c67-433f-4a02-b04c-175cceb37de6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47d365c0801c405f9a7384ed2ca5bfb5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:06.563206 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-60d03c67-433f-4a02-b04c-175cceb37de6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47d365c0801c405f9a7384ed2ca5bfb5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:06.570862 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-60d03c67-433f-4a02-b04c-175cceb37de6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47d365c0801c405f9a7384ed2ca5bfb5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:06.583662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-60d03c67-433f-4a02-b04c-175cceb37de6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47d365c0801c405f9a7384ed2ca5bfb5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:06.588861 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63935784-1275-4aad-87f6-5a9a6fc86e15 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:06.589408 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63935784-1275-4aad-87f6-5a9a6fc86e15 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:06.589408 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63935784-1275-4aad-87f6-5a9a6fc86e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:06.609238 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-63935784-1275-4aad-87f6-5a9a6fc86e15 None None] MFA Rules not processed for user `f289a1bdd79b4ba0bf9499edb7729c9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:09:06.611119 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-63935784-1275-4aad-87f6-5a9a6fc86e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:09:06.618344 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1203/2418] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:09:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:09:06.678072 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1216/2419] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:09:06 2022] DELETE /identity/v3/users/47d365c0801c405f9a7384ed2ca5bfb5 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:09:06.682694 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805c1f4-df99-4a5c-b596-275dbd5a24ef None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:06.683106 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805c1f4-df99-4a5c-b596-275dbd5a24ef None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:06.683396 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805c1f4-df99-4a5c-b596-275dbd5a24ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:06.707883 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-2805c1f4-df99-4a5c-b596-275dbd5a24ef None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:09:06.709131 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2805c1f4-df99-4a5c-b596-275dbd5a24ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:09:06.774641 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1204/2420] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:09:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:09:06.785416 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d03c67-433f-4a02-b04c-175cceb37de6 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:06.786802 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-60d03c67-433f-4a02-b04c-175cceb37de6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:09:06.857833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d03c67-433f-4a02-b04c-175cceb37de6 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:06.864902 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf768a44-9a41-4ee5-bf2e-0aa89b8106b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:06.865526 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf768a44-9a41-4ee5-bf2e-0aa89b8106b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:06.865808 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf768a44-9a41-4ee5-bf2e-0aa89b8106b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:06.866086 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf768a44-9a41-4ee5-bf2e-0aa89b8106b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:06.875838 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf768a44-9a41-4ee5-bf2e-0aa89b8106b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:06.877334 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf768a44-9a41-4ee5-bf2e-0aa89b8106b9 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:06.887621 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1217/2421] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:09:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:09:07.326949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805c1f4-df99-4a5c-b596-275dbd5a24ef None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:07.328416 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2805c1f4-df99-4a5c-b596-275dbd5a24ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:09:07.341304 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805c1f4-df99-4a5c-b596-275dbd5a24ef None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:07.346308 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71393b46-02bf-4d6a-ad63-dce2e79a7823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:07.347008 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71393b46-02bf-4d6a-ad63-dce2e79a7823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:07.347360 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71393b46-02bf-4d6a-ad63-dce2e79a7823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:07.347699 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71393b46-02bf-4d6a-ad63-dce2e79a7823 admin admin] PATH_INFO: `/v3/projects/984a72f1d3564c31aaef8970b9f3c077` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:07.349442 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71393b46-02bf-4d6a-ad63-dce2e79a7823 admin admin] RBAC: Authorizing `identity:delete_project(project_id=984a72f1d3564c31aaef8970b9f3c077)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:07.350958 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71393b46-02bf-4d6a-ad63-dce2e79a7823 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:07.473602 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-71393b46-02bf-4d6a-ad63-dce2e79a7823 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '984a72f1d3564c31aaef8970b9f3c077', 'request_id': 'req-71393b46-02bf-4d6a-ad63-dce2e79a7823'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:07.563841 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-71393b46-02bf-4d6a-ad63-dce2e79a7823 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '984a72f1d3564c31aaef8970b9f3c077', 'request_id': 'req-71393b46-02bf-4d6a-ad63-dce2e79a7823'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:07.572704 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-71393b46-02bf-4d6a-ad63-dce2e79a7823 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '984a72f1d3564c31aaef8970b9f3c077', 'request_id': 'req-71393b46-02bf-4d6a-ad63-dce2e79a7823'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:07.582711 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-71393b46-02bf-4d6a-ad63-dce2e79a7823 admin admin] The token cache is being invalidate because project 984a72f1d3564c31aaef8970b9f3c077 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:09:07.583010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-71393b46-02bf-4d6a-ad63-dce2e79a7823 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:07.583686 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-71393b46-02bf-4d6a-ad63-dce2e79a7823 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:07.585190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1205/2422] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:09:07 2022] DELETE /identity/v3/projects/984a72f1d3564c31aaef8970b9f3c077 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:09:07.589647 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf768a44-9a41-4ee5-bf2e-0aa89b8106b9 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:07.603311 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf768a44-9a41-4ee5-bf2e-0aa89b8106b9 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:07.622624 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:09:07.663065 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b454e6bf-ca8a-451c-a621-34593ca1a85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:07.663705 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454e6bf-ca8a-451c-a621-34593ca1a85a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:07.664008 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454e6bf-ca8a-451c-a621-34593ca1a85a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:07.664308 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454e6bf-ca8a-451c-a621-34593ca1a85a admin admin] PATH_INFO: `/v3/users/ea5cceb96da644bdb2c4b4b77cc46635` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:07.665419 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b454e6bf-ca8a-451c-a621-34593ca1a85a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea5cceb96da644bdb2c4b4b77cc46635)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:07.667403 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b454e6bf-ca8a-451c-a621-34593ca1a85a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:07.804718 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b454e6bf-ca8a-451c-a621-34593ca1a85a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea5cceb96da644bdb2c4b4b77cc46635'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:07.805366 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b454e6bf-ca8a-451c-a621-34593ca1a85a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea5cceb96da644bdb2c4b4b77cc46635'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:07.805984 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b454e6bf-ca8a-451c-a621-34593ca1a85a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea5cceb96da644bdb2c4b4b77cc46635'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:07.816189 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b454e6bf-ca8a-451c-a621-34593ca1a85a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea5cceb96da644bdb2c4b4b77cc46635'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:07.832425 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b454e6bf-ca8a-451c-a621-34593ca1a85a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea5cceb96da644bdb2c4b4b77cc46635'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:07.934519 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1218/2423] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:09:07 2022] DELETE /identity/v3/users/ea5cceb96da644bdb2c4b4b77cc46635 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:09:08.424041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71393b46-02bf-4d6a-ad63-dce2e79a7823 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:08.425962 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-71393b46-02bf-4d6a-ad63-dce2e79a7823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:09:08.446017 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71393b46-02bf-4d6a-ad63-dce2e79a7823 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:08.509919 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f20b93-97ea-47c6-b03b-1cf7e8ddf75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:08.510551 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f20b93-97ea-47c6-b03b-1cf7e8ddf75f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:08.510832 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f20b93-97ea-47c6-b03b-1cf7e8ddf75f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:08.511103 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f20b93-97ea-47c6-b03b-1cf7e8ddf75f admin admin] PATH_INFO: `/v3/projects/aa7e936295b5471ca36999c614bed29c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:08.512375 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f20b93-97ea-47c6-b03b-1cf7e8ddf75f admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa7e936295b5471ca36999c614bed29c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:08.514323 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f20b93-97ea-47c6-b03b-1cf7e8ddf75f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:08.547709 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:09:08.647297 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-60f20b93-97ea-47c6-b03b-1cf7e8ddf75f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa7e936295b5471ca36999c614bed29c', 'request_id': 'req-60f20b93-97ea-47c6-b03b-1cf7e8ddf75f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:08.724448 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-60f20b93-97ea-47c6-b03b-1cf7e8ddf75f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa7e936295b5471ca36999c614bed29c', 'request_id': 'req-60f20b93-97ea-47c6-b03b-1cf7e8ddf75f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:08.733340 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-60f20b93-97ea-47c6-b03b-1cf7e8ddf75f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa7e936295b5471ca36999c614bed29c', 'request_id': 'req-60f20b93-97ea-47c6-b03b-1cf7e8ddf75f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:08.747466 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-60f20b93-97ea-47c6-b03b-1cf7e8ddf75f admin admin] The token cache is being invalidate because project aa7e936295b5471ca36999c614bed29c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:09:08.747885 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-60f20b93-97ea-47c6-b03b-1cf7e8ddf75f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:08.748668 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-60f20b93-97ea-47c6-b03b-1cf7e8ddf75f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:08.750407 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1206/2424] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:09:08 2022] DELETE /identity/v3/projects/aa7e936295b5471ca36999c614bed29c => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:09:08.755751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e9733-934e-47ce-9f2d-92c628269af3 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:08.755751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e9733-934e-47ce-9f2d-92c628269af3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:08.755751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e9733-934e-47ce-9f2d-92c628269af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:08.777795 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-783e9733-934e-47ce-9f2d-92c628269af3 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:09:08.779009 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-783e9733-934e-47ce-9f2d-92c628269af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:09:08.854709 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1219/2425] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:09:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:09:08.863626 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eedf50-446a-465f-bd92-0ecb38c284c3 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:08.863940 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eedf50-446a-465f-bd92-0ecb38c284c3 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:08.864253 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eedf50-446a-465f-bd92-0ecb38c284c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:08.884329 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-55eedf50-446a-465f-bd92-0ecb38c284c3 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:09:08.885500 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-55eedf50-446a-465f-bd92-0ecb38c284c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:09:08.893304 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1207/2426] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:09:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:09:08.897596 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e9733-934e-47ce-9f2d-92c628269af3 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:08.907525 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e9733-934e-47ce-9f2d-92c628269af3 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:08.912631 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65352e2-7b3c-4056-b38f-23c485358b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:08.913271 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65352e2-7b3c-4056-b38f-23c485358b97 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:08.913574 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65352e2-7b3c-4056-b38f-23c485358b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:08.913853 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65352e2-7b3c-4056-b38f-23c485358b97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:08.914510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65352e2-7b3c-4056-b38f-23c485358b97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:09:08.914842 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65352e2-7b3c-4056-b38f-23c485358b97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:08.916445 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65352e2-7b3c-4056-b38f-23c485358b97 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:08.928867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1220/2427] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:09:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:09:08.933193 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eedf50-446a-465f-bd92-0ecb38c284c3 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:08.942927 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eedf50-446a-465f-bd92-0ecb38c284c3 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:08.947639 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c1ece-e5c5-4767-a008-55b69dcc2cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:08.948261 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c1ece-e5c5-4767-a008-55b69dcc2cb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:08.948542 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c1ece-e5c5-4767-a008-55b69dcc2cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:08.948809 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c1ece-e5c5-4767-a008-55b69dcc2cb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:08.949471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c1ece-e5c5-4767-a008-55b69dcc2cb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:08.951346 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c1ece-e5c5-4767-a008-55b69dcc2cb5 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:09.057435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1208/2428] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:09:08 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:09:09.061612 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65352e2-7b3c-4056-b38f-23c485358b97 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:09.071701 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65352e2-7b3c-4056-b38f-23c485358b97 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:09.097955 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:09:09.129227 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a54b94-592c-43b7-b963-d1fbf5724681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:09.129832 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a54b94-592c-43b7-b963-d1fbf5724681 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:09.130110 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a54b94-592c-43b7-b963-d1fbf5724681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:09.130378 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a54b94-592c-43b7-b963-d1fbf5724681 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:09.131001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a54b94-592c-43b7-b963-d1fbf5724681 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:09.132951 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a54b94-592c-43b7-b963-d1fbf5724681 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:09.239021 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1221/2429] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:09:09 2022] POST /identity/v3/users => generated 294 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:09:09.242956 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c1ece-e5c5-4767-a008-55b69dcc2cb5 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:09.253149 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c1ece-e5c5-4767-a008-55b69dcc2cb5 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:09.257827 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba61ff-2436-478c-9f4e-df9d8d16b133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:09.258448 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba61ff-2436-478c-9f4e-df9d8d16b133 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:09.258732 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba61ff-2436-478c-9f4e-df9d8d16b133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:09.259007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba61ff-2436-478c-9f4e-df9d8d16b133 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:09.259648 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba61ff-2436-478c-9f4e-df9d8d16b133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:09.261271 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba61ff-2436-478c-9f4e-df9d8d16b133 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:09.270800 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1209/2430] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:09:09.280789 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a54b94-592c-43b7-b963-d1fbf5724681 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:09.291276 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a54b94-592c-43b7-b963-d1fbf5724681 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:09.298152 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13bd5fc-d7d6-4b98-962f-a0635baf7442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:09.298751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13bd5fc-d7d6-4b98-962f-a0635baf7442 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:09.299029 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13bd5fc-d7d6-4b98-962f-a0635baf7442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:09.299302 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13bd5fc-d7d6-4b98-962f-a0635baf7442 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:09.299944 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13bd5fc-d7d6-4b98-962f-a0635baf7442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:09.301971 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13bd5fc-d7d6-4b98-962f-a0635baf7442 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:09.313312 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1222/2431] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:09:09.317407 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba61ff-2436-478c-9f4e-df9d8d16b133 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:09.323326 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13bd5fc-d7d6-4b98-962f-a0635baf7442 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:09.323326 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e13bd5fc-d7d6-4b98-962f-a0635baf7442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:09:09.328051 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba61ff-2436-478c-9f4e-df9d8d16b133 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:09.332927 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f8ef9-f2e9-4f1a-9614-c4fda2b1f531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:09.333572 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f8ef9-f2e9-4f1a-9614-c4fda2b1f531 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:09.333848 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f8ef9-f2e9-4f1a-9614-c4fda2b1f531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:09.334118 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f8ef9-f2e9-4f1a-9614-c4fda2b1f531 admin admin] PATH_INFO: `/v3/projects/596c3ab232d64522b7cf774a7e08e22a/users/b6d2c0a873f744278cad2168cad6e4b6/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:09.349823 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889f8ef9-f2e9-4f1a-9614-c4fda2b1f531 admin admin] RBAC: Authorizing `identity:create_grant(project_id=596c3ab232d64522b7cf774a7e08e22a, user_id=b6d2c0a873f744278cad2168cad6e4b6, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:09.352735 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889f8ef9-f2e9-4f1a-9614-c4fda2b1f531 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:09.389642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13bd5fc-d7d6-4b98-962f-a0635baf7442 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:09.394772 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32edd91d-360b-4feb-acf1-4e3bfd4f608a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4f6de2065d4fbea3fa141da760f708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:09.395366 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32edd91d-360b-4feb-acf1-4e3bfd4f608a service placement] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:09.395645 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32edd91d-360b-4feb-acf1-4e3bfd4f608a service placement] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:09.395919 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32edd91d-360b-4feb-acf1-4e3bfd4f608a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:09.397751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-32edd91d-360b-4feb-acf1-4e3bfd4f608a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:09:09.409690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32edd91d-360b-4feb-acf1-4e3bfd4f608a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:09.411318 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32edd91d-360b-4feb-acf1-4e3bfd4f608a service placement] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:09.421015 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1223/2432] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:09:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:09:09.439235 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1210/2433] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:09:09 2022] PUT /identity/v3/projects/596c3ab232d64522b7cf774a7e08e22a/users/b6d2c0a873f744278cad2168cad6e4b6/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:09:09.451112 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da9c1a-00d9-4a07-9fff-bf4e659d56ab None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:09.451400 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da9c1a-00d9-4a07-9fff-bf4e659d56ab None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:09.451670 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da9c1a-00d9-4a07-9fff-bf4e659d56ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:09.479832 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c8da9c1a-00d9-4a07-9fff-bf4e659d56ab None None] MFA Rules not processed for user `b6d2c0a873f744278cad2168cad6e4b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:09:09.481106 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c8da9c1a-00d9-4a07-9fff-bf4e659d56ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:09:09.545431 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1224/2434] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:09:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:09:09.549808 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f8ef9-f2e9-4f1a-9614-c4fda2b1f531 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:09.611065 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f8ef9-f2e9-4f1a-9614-c4fda2b1f531 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:09.615893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b667bd-fbff-4175-a626-6fc03fbf129c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:09.616526 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b667bd-fbff-4175-a626-6fc03fbf129c admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:09.616807 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b667bd-fbff-4175-a626-6fc03fbf129c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:09.617079 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b667bd-fbff-4175-a626-6fc03fbf129c admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:09.617708 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b667bd-fbff-4175-a626-6fc03fbf129c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:09.619599 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b667bd-fbff-4175-a626-6fc03fbf129c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:09.697370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1211/2435] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:09:09 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:09:09.701950 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da9c1a-00d9-4a07-9fff-bf4e659d56ab None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:09.713171 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da9c1a-00d9-4a07-9fff-bf4e659d56ab None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:09.745519 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:09:09.778386 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862f808-81f4-409a-a55b-f9bc9a95faac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:09.779084 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862f808-81f4-409a-a55b-f9bc9a95faac admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:09.779370 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862f808-81f4-409a-a55b-f9bc9a95faac admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:09.779822 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862f808-81f4-409a-a55b-f9bc9a95faac admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:09.781067 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d862f808-81f4-409a-a55b-f9bc9a95faac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:09.784315 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d862f808-81f4-409a-a55b-f9bc9a95faac admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:09.891188 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1225/2436] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:09:09 2022] POST /identity/v3/users => generated 299 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:09:09.895042 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b667bd-fbff-4175-a626-6fc03fbf129c admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:09.906121 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b667bd-fbff-4175-a626-6fc03fbf129c admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:09.910988 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f26ba-4dec-4662-a51a-851d15dc94d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:09.914910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f26ba-4dec-4662-a51a-851d15dc94d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:09.914910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f26ba-4dec-4662-a51a-851d15dc94d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:09.914910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f26ba-4dec-4662-a51a-851d15dc94d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:09.914910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2f26ba-4dec-4662-a51a-851d15dc94d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:09.914910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2f26ba-4dec-4662-a51a-851d15dc94d3 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:09.928063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1212/2437] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:09:09.932726 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862f808-81f4-409a-a55b-f9bc9a95faac admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:09.944056 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862f808-81f4-409a-a55b-f9bc9a95faac admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:09.950257 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a3261-7b1d-41e0-89ee-5219d5e0c90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:09.951014 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a3261-7b1d-41e0-89ee-5219d5e0c90b admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:09.952168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a3261-7b1d-41e0-89ee-5219d5e0c90b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:09.952168 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a3261-7b1d-41e0-89ee-5219d5e0c90b admin admin] PATH_INFO: `/v3/projects/7aecaf658b4249a5b8ed780ab741854d/users/02f0912881934fdf93eafb042f8932f3/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:09.968620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a3261-7b1d-41e0-89ee-5219d5e0c90b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aecaf658b4249a5b8ed780ab741854d, user_id=02f0912881934fdf93eafb042f8932f3, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:09.972288 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a3261-7b1d-41e0-89ee-5219d5e0c90b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:10.050794 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1226/2438] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:09:09 2022] PUT /identity/v3/projects/7aecaf658b4249a5b8ed780ab741854d/users/02f0912881934fdf93eafb042f8932f3/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:09:10.055763 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f26ba-4dec-4662-a51a-851d15dc94d3 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:10.066690 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f26ba-4dec-4662-a51a-851d15dc94d3 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:10.080022 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:09:10.123146 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92171ac-ab13-44e0-b0b0-c2bc440833f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:10.123662 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92171ac-ab13-44e0-b0b0-c2bc440833f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:10.123942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92171ac-ab13-44e0-b0b0-c2bc440833f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:10.124262 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92171ac-ab13-44e0-b0b0-c2bc440833f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:10.124896 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92171ac-ab13-44e0-b0b0-c2bc440833f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:10.126479 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92171ac-ab13-44e0-b0b0-c2bc440833f0 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:10.137032 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1213/2439] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:09:10 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:09:10.141219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a3261-7b1d-41e0-89ee-5219d5e0c90b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:10.151722 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a3261-7b1d-41e0-89ee-5219d5e0c90b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:10.156386 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2c8a2-13f2-4c95-9651-216e27d86d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:10.157016 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2c8a2-13f2-4c95-9651-216e27d86d9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:10.163079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2c8a2-13f2-4c95-9651-216e27d86d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:10.163079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2c8a2-13f2-4c95-9651-216e27d86d9f admin admin] PATH_INFO: `/v3/projects/7aecaf658b4249a5b8ed780ab741854d/users/02f0912881934fdf93eafb042f8932f3/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:10.163079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c2c8a2-13f2-4c95-9651-216e27d86d9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aecaf658b4249a5b8ed780ab741854d, user_id=02f0912881934fdf93eafb042f8932f3, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:10.163079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c2c8a2-13f2-4c95-9651-216e27d86d9f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:10.172773 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1227/2440] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:09:10 2022] PUT /identity/v3/projects/7aecaf658b4249a5b8ed780ab741854d/users/02f0912881934fdf93eafb042f8932f3/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:09:10.182843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844b902-d4eb-478a-aabe-a743bcd7c038 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:10.183202 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844b902-d4eb-478a-aabe-a743bcd7c038 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:10.183547 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844b902-d4eb-478a-aabe-a743bcd7c038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:10.211919 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-5844b902-d4eb-478a-aabe-a743bcd7c038 None None] MFA Rules not processed for user `02f0912881934fdf93eafb042f8932f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:09:10.213162 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5844b902-d4eb-478a-aabe-a743bcd7c038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:09:10.280444 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1214/2441] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:09:10 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:09:10.291114 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2c8a2-13f2-4c95-9651-216e27d86d9f admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:10.292907 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f1c2c8a2-13f2-4c95-9651-216e27d86d9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:09:10.350272 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2c8a2-13f2-4c95-9651-216e27d86d9f admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:10.353824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd91e8-8238-4942-93fc-b4d4c5125442 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:10.354417 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd91e8-8238-4942-93fc-b4d4c5125442 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:10.354691 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd91e8-8238-4942-93fc-b4d4c5125442 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:10.354960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd91e8-8238-4942-93fc-b4d4c5125442 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:10.364129 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bd91e8-8238-4942-93fc-b4d4c5125442 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:10.365549 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bd91e8-8238-4942-93fc-b4d4c5125442 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:10.375305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1228/2442] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:09:10 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:09:11.392054 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5844b902-d4eb-478a-aabe-a743bcd7c038 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:11.453465 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5844b902-d4eb-478a-aabe-a743bcd7c038 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:11.458243 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed9966-b990-4a0a-a37f-a055577019eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:11.458863 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed9966-b990-4a0a-a37f-a055577019eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:11.459141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed9966-b990-4a0a-a37f-a055577019eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:11.459417 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed9966-b990-4a0a-a37f-a055577019eb admin admin] PATH_INFO: `/v3/users/b6d2c0a873f744278cad2168cad6e4b6` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:11.460545 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ed9966-b990-4a0a-a37f-a055577019eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6d2c0a873f744278cad2168cad6e4b6)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:11.462458 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ed9966-b990-4a0a-a37f-a055577019eb admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:11.677171 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:09:11.682208 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e8ed9966-b990-4a0a-a37f-a055577019eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6d2c0a873f744278cad2168cad6e4b6'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:11.682962 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e8ed9966-b990-4a0a-a37f-a055577019eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6d2c0a873f744278cad2168cad6e4b6'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:11.690954 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e8ed9966-b990-4a0a-a37f-a055577019eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6d2c0a873f744278cad2168cad6e4b6'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:11.702985 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e8ed9966-b990-4a0a-a37f-a055577019eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6d2c0a873f744278cad2168cad6e4b6'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:11.705255 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e8ed9966-b990-4a0a-a37f-a055577019eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6d2c0a873f744278cad2168cad6e4b6'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:11.792729 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1215/2443] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:09:11 2022] DELETE /identity/v3/users/b6d2c0a873f744278cad2168cad6e4b6 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:09:11.797707 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20526332-b059-449c-b974-038b1ec977cd None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:11.798190 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20526332-b059-449c-b974-038b1ec977cd None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:11.801622 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20526332-b059-449c-b974-038b1ec977cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:11.821613 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-20526332-b059-449c-b974-038b1ec977cd None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:09:11.822815 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-20526332-b059-449c-b974-038b1ec977cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:09:11.882120 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1229/2444] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:09:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:09:11.891592 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed9966-b990-4a0a-a37f-a055577019eb admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:11.893108 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e8ed9966-b990-4a0a-a37f-a055577019eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:09:11.950253 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed9966-b990-4a0a-a37f-a055577019eb admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:11.954039 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485bca93-25c8-4475-a664-17cb412f68a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:11.954640 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bca93-25c8-4475-a664-17cb412f68a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:11.954922 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bca93-25c8-4475-a664-17cb412f68a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:11.955194 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bca93-25c8-4475-a664-17cb412f68a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:11.964062 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485bca93-25c8-4475-a664-17cb412f68a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:11.966018 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485bca93-25c8-4475-a664-17cb412f68a4 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:11.975441 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1216/2445] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:09:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:09:12.349239 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20526332-b059-449c-b974-038b1ec977cd None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:12.353082 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-20526332-b059-449c-b974-038b1ec977cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:09:12.365598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20526332-b059-449c-b974-038b1ec977cd None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:12.371559 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc165e0f-ac7a-41af-9d09-d3da07c111aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:12.372498 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc165e0f-ac7a-41af-9d09-d3da07c111aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:12.372872 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc165e0f-ac7a-41af-9d09-d3da07c111aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:12.373268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc165e0f-ac7a-41af-9d09-d3da07c111aa admin admin] PATH_INFO: `/v3/projects/596c3ab232d64522b7cf774a7e08e22a` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:12.374714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc165e0f-ac7a-41af-9d09-d3da07c111aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=596c3ab232d64522b7cf774a7e08e22a)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:12.376863 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc165e0f-ac7a-41af-9d09-d3da07c111aa admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:12.397360 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:09:12.501500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fc165e0f-ac7a-41af-9d09-d3da07c111aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '596c3ab232d64522b7cf774a7e08e22a', 'request_id': 'req-fc165e0f-ac7a-41af-9d09-d3da07c111aa'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:12.507755 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fc165e0f-ac7a-41af-9d09-d3da07c111aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '596c3ab232d64522b7cf774a7e08e22a', 'request_id': 'req-fc165e0f-ac7a-41af-9d09-d3da07c111aa'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:12.580785 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fc165e0f-ac7a-41af-9d09-d3da07c111aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '596c3ab232d64522b7cf774a7e08e22a', 'request_id': 'req-fc165e0f-ac7a-41af-9d09-d3da07c111aa'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:12.591865 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fc165e0f-ac7a-41af-9d09-d3da07c111aa admin admin] The token cache is being invalidate because project 596c3ab232d64522b7cf774a7e08e22a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:09:12.592182 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fc165e0f-ac7a-41af-9d09-d3da07c111aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:12.592803 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fc165e0f-ac7a-41af-9d09-d3da07c111aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:12.594276 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1230/2446] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:09:12 2022] DELETE /identity/v3/projects/596c3ab232d64522b7cf774a7e08e22a => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:09:12.598463 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485bca93-25c8-4475-a664-17cb412f68a4 service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:12.608159 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485bca93-25c8-4475-a664-17cb412f68a4 service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:12.662994 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768267d9-71d3-4585-8a33-38a16269ee5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:12.663641 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768267d9-71d3-4585-8a33-38a16269ee5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:12.663921 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768267d9-71d3-4585-8a33-38a16269ee5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:12.664232 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768267d9-71d3-4585-8a33-38a16269ee5e admin admin] PATH_INFO: `/v3/users/02f0912881934fdf93eafb042f8932f3` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:12.665357 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768267d9-71d3-4585-8a33-38a16269ee5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=02f0912881934fdf93eafb042f8932f3)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:12.667266 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768267d9-71d3-4585-8a33-38a16269ee5e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:12.863386 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-768267d9-71d3-4585-8a33-38a16269ee5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02f0912881934fdf93eafb042f8932f3'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:12.864033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-768267d9-71d3-4585-8a33-38a16269ee5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02f0912881934fdf93eafb042f8932f3'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:12.870876 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-768267d9-71d3-4585-8a33-38a16269ee5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02f0912881934fdf93eafb042f8932f3'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:12.876633 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:09:12.883625 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-768267d9-71d3-4585-8a33-38a16269ee5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02f0912881934fdf93eafb042f8932f3'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:12.890245 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-768267d9-71d3-4585-8a33-38a16269ee5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02f0912881934fdf93eafb042f8932f3'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:12.936517 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1217/2447] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:09:12 2022] DELETE /identity/v3/users/02f0912881934fdf93eafb042f8932f3 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:09:13.343299 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc165e0f-ac7a-41af-9d09-d3da07c111aa admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:13.344950 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fc165e0f-ac7a-41af-9d09-d3da07c111aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:09:13.360343 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc165e0f-ac7a-41af-9d09-d3da07c111aa admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:13.417874 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce1790-0887-45a7-9d22-5f819d965d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:13.419071 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce1790-0887-45a7-9d22-5f819d965d41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:13.419420 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce1790-0887-45a7-9d22-5f819d965d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:13.419665 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce1790-0887-45a7-9d22-5f819d965d41 admin admin] PATH_INFO: `/v3/projects/7aecaf658b4249a5b8ed780ab741854d` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:13.420785 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ce1790-0887-45a7-9d22-5f819d965d41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7aecaf658b4249a5b8ed780ab741854d)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:13.427029 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ce1790-0887-45a7-9d22-5f819d965d41 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:13.520425 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e4ce1790-0887-45a7-9d22-5f819d965d41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7aecaf658b4249a5b8ed780ab741854d', 'request_id': 'req-e4ce1790-0887-45a7-9d22-5f819d965d41'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:13.526999 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e4ce1790-0887-45a7-9d22-5f819d965d41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7aecaf658b4249a5b8ed780ab741854d', 'request_id': 'req-e4ce1790-0887-45a7-9d22-5f819d965d41'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:13.615927 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e4ce1790-0887-45a7-9d22-5f819d965d41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7aecaf658b4249a5b8ed780ab741854d', 'request_id': 'req-e4ce1790-0887-45a7-9d22-5f819d965d41'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:13.627667 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e4ce1790-0887-45a7-9d22-5f819d965d41 admin admin] The token cache is being invalidate because project 7aecaf658b4249a5b8ed780ab741854d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:09:13.628041 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e4ce1790-0887-45a7-9d22-5f819d965d41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:13.628614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e4ce1790-0887-45a7-9d22-5f819d965d41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:09:13.630206 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1231/2448] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:09:13 2022] DELETE /identity/v3/projects/7aecaf658b4249a5b8ed780ab741854d => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:09:13.635028 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f0381-33c6-4b86-b9f8-d8856083d6bf None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:13.635505 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f0381-33c6-4b86-b9f8-d8856083d6bf None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:13.635600 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f0381-33c6-4b86-b9f8-d8856083d6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:13.656289 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-f26f0381-33c6-4b86-b9f8-d8856083d6bf None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:09:13.657508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f26f0381-33c6-4b86-b9f8-d8856083d6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:09:13.720862 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1218/2449] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:09:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:09:13.728522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f6112-49ca-4814-96e4-4810261e1c2a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:13.728841 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f6112-49ca-4814-96e4-4810261e1c2a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:13.729118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f6112-49ca-4814-96e4-4810261e1c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:13.750868 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-7d2f6112-49ca-4814-96e4-4810261e1c2a None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:09:13.752337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7d2f6112-49ca-4814-96e4-4810261e1c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:09:13.760504 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1232/2450] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:09:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:09:13.764909 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26f0381-33c6-4b86-b9f8-d8856083d6bf None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:13.775177 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26f0381-33c6-4b86-b9f8-d8856083d6bf None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:13.780652 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0da59-9795-4473-9786-dfde2e3ef20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:13.781252 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0da59-9795-4473-9786-dfde2e3ef20a admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:13.781532 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0da59-9795-4473-9786-dfde2e3ef20a admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:13.781808 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0da59-9795-4473-9786-dfde2e3ef20a admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:13.782460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0da59-9795-4473-9786-dfde2e3ef20a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:09:13.782764 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0da59-9795-4473-9786-dfde2e3ef20a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:13.784401 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0da59-9795-4473-9786-dfde2e3ef20a admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:13.796878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1219/2451] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:09:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:09:13.801220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f6112-49ca-4814-96e4-4810261e1c2a None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:13.810715 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f6112-49ca-4814-96e4-4810261e1c2a None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:13.815458 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d883330-2b09-4166-afd9-707f85410fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:13.818292 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d883330-2b09-4166-afd9-707f85410fcb admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:13.818292 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d883330-2b09-4166-afd9-707f85410fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:13.818292 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d883330-2b09-4166-afd9-707f85410fcb admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:13.818292 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d883330-2b09-4166-afd9-707f85410fcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:13.821780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d883330-2b09-4166-afd9-707f85410fcb admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:13.903535 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1233/2452] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:09:13 2022] POST /identity/v3/projects => generated 371 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:09:13.909672 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0da59-9795-4473-9786-dfde2e3ef20a admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:13.919454 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0da59-9795-4473-9786-dfde2e3ef20a admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:13.977031 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1849b3-abc2-4605-9c80-71c4c5c202a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:13.977635 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1849b3-abc2-4605-9c80-71c4c5c202a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:13.977908 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1849b3-abc2-4605-9c80-71c4c5c202a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:13.978176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1849b3-abc2-4605-9c80-71c4c5c202a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:13.978799 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1849b3-abc2-4605-9c80-71c4c5c202a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:13.980748 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1849b3-abc2-4605-9c80-71c4c5c202a7 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:14.084542 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1220/2453] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:09:13 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:09:14.088627 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d883330-2b09-4166-afd9-707f85410fcb admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:14.099515 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d883330-2b09-4166-afd9-707f85410fcb admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:14.104193 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb49f0-2467-47f5-8ed2-5fe34d78333c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:14.104792 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb49f0-2467-47f5-8ed2-5fe34d78333c admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:14.105072 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb49f0-2467-47f5-8ed2-5fe34d78333c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:14.105404 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb49f0-2467-47f5-8ed2-5fe34d78333c admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:14.106066 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb49f0-2467-47f5-8ed2-5fe34d78333c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:14.107658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb49f0-2467-47f5-8ed2-5fe34d78333c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:14.108913 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:09:14.118097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1234/2454] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:09:14.122184 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1849b3-abc2-4605-9c80-71c4c5c202a7 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:14.132278 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1849b3-abc2-4605-9c80-71c4c5c202a7 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:14.137615 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9d608-25b9-4457-ac2a-141971b96634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:14.138209 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9d608-25b9-4457-ac2a-141971b96634 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:14.138495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9d608-25b9-4457-ac2a-141971b96634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:14.138770 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9d608-25b9-4457-ac2a-141971b96634 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:14.139413 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b9d608-25b9-4457-ac2a-141971b96634 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:14.141072 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b9d608-25b9-4457-ac2a-141971b96634 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:14.151489 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1221/2455] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:09:14.156279 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb49f0-2467-47f5-8ed2-5fe34d78333c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:14.166079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb49f0-2467-47f5-8ed2-5fe34d78333c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:14.171288 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aee546-9c86-4213-a542-81b0342c202b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:14.171922 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aee546-9c86-4213-a542-81b0342c202b admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:14.172243 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aee546-9c86-4213-a542-81b0342c202b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:14.172536 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aee546-9c86-4213-a542-81b0342c202b admin admin] PATH_INFO: `/v3/projects/0ac874ab0dff440e88da50892a1d2089/users/302fe3b65b8944219f6bc27690ab82e4/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:14.187398 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aee546-9c86-4213-a542-81b0342c202b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ac874ab0dff440e88da50892a1d2089, user_id=302fe3b65b8944219f6bc27690ab82e4, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:14.190063 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aee546-9c86-4213-a542-81b0342c202b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:14.256467 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1235/2456] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:09:14 2022] PUT /identity/v3/projects/0ac874ab0dff440e88da50892a1d2089/users/302fe3b65b8944219f6bc27690ab82e4/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:09:14.325074 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc3614-40cd-464a-b8f8-0876dd1f98c9 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:14.325394 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc3614-40cd-464a-b8f8-0876dd1f98c9 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:14.325615 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc3614-40cd-464a-b8f8-0876dd1f98c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:14.353940 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-acbc3614-40cd-464a-b8f8-0876dd1f98c9 None None] MFA Rules not processed for user `302fe3b65b8944219f6bc27690ab82e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:09:14.355176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-acbc3614-40cd-464a-b8f8-0876dd1f98c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:09:14.389000 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:09:14.423025 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1222/2457] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:09:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:09:14.434399 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aee546-9c86-4213-a542-81b0342c202b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:14.436301 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e0aee546-9c86-4213-a542-81b0342c202b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:09:14.514545 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aee546-9c86-4213-a542-81b0342c202b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:14.518275 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb4666-9148-4179-9e02-96420c7d6a42 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7832e97d7e97439e821268d05816ae3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:14.518878 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb4666-9148-4179-9e02-96420c7d6a42 service glance] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:14.519160 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb4666-9148-4179-9e02-96420c7d6a42 service glance] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:14.519432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb4666-9148-4179-9e02-96420c7d6a42 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:14.527837 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bb4666-9148-4179-9e02-96420c7d6a42 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:14.529313 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bb4666-9148-4179-9e02-96420c7d6a42 service glance] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:14.537958 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1236/2458] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:09:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:09:24.534176 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=47,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:09:24.558156 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=11 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:09:48.612458 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4b369-f1ff-40d8-8053-dfd72bf0e4e8 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:48.613096 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4b369-f1ff-40d8-8053-dfd72bf0e4e8 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:48.613096 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4b369-f1ff-40d8-8053-dfd72bf0e4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:48.636197 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-d1e4b369-f1ff-40d8-8053-dfd72bf0e4e8 None None] MFA Rules not processed for user `8c21ea7d3cbb494886e60a4bd4f39145`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:09:48.637742 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d1e4b369-f1ff-40d8-8053-dfd72bf0e4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:09:48.712661 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1223/2459] 10.222.0.48 () {58 vars in 1009 bytes} [Fri Sep 9 21:09:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:09:48.769838 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb4666-9148-4179-9e02-96420c7d6a42 service glance] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:48.772147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-12bb4666-9148-4179-9e02-96420c7d6a42 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:09:48.795455 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4b369-f1ff-40d8-8053-dfd72bf0e4e8 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:09:48.796889 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d1e4b369-f1ff-40d8-8053-dfd72bf0e4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:09:48.848310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb4666-9148-4179-9e02-96420c7d6a42 service glance] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:48.853528 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038eed94-1ac0-4809-b6f2-e35965f1f360 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d28fa5d49d44120bd5eaec2d4e5fdbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:48.854140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038eed94-1ac0-4809-b6f2-e35965f1f360 service swift] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:48.854421 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038eed94-1ac0-4809-b6f2-e35965f1f360 service swift] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:48.854698 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038eed94-1ac0-4809-b6f2-e35965f1f360 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:48.857318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4b369-f1ff-40d8-8053-dfd72bf0e4e8 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:09:48.861648 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de7b9c-7924-4d37-a6cf-35c60c9b071a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:09:48.862252 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de7b9c-7924-4d37-a6cf-35c60c9b071a service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:09:48.862532 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de7b9c-7924-4d37-a6cf-35c60c9b071a service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:09:48.862799 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de7b9c-7924-4d37-a6cf-35c60c9b071a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:09:48.863139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038eed94-1ac0-4809-b6f2-e35965f1f360 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:48.864638 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b5de7b9c-7924-4d37-a6cf-35c60c9b071a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:09:48.864824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038eed94-1ac0-4809-b6f2-e35965f1f360 service swift] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:48.873149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1237/2460] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Sep 9 21:09:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 21:09:48.982367 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5de7b9c-7924-4d37-a6cf-35c60c9b071a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:09:48.984305 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5de7b9c-7924-4d37-a6cf-35c60c9b071a service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:09:48.999719 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1224/2461] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:09:48 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:09:58.869749 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=32 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:09:58.993537 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=75 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:10:10.287253 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d5b68-0c6d-4349-8fc3-f862e5cc992b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:10.287806 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d5b68-0c6d-4349-8fc3-f862e5cc992b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:10.287806 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d5b68-0c6d-4349-8fc3-f862e5cc992b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:10.306327 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-986d5b68-0c6d-4349-8fc3-f862e5cc992b None None] MFA Rules not processed for user `0d9129eaa2694ef6b3f006b67ef88c52`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:10:10.307647 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-986d5b68-0c6d-4349-8fc3-f862e5cc992b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:10:10.367586 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1238/2462] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:10:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:10:12.857555 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de7b9c-7924-4d37-a6cf-35c60c9b071a service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:10:12.926363 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de7b9c-7924-4d37-a6cf-35c60c9b071a service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:10:12.931186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4540054-5974-4ff9-bde0-838354fb6860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:10:12.931818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4540054-5974-4ff9-bde0-838354fb6860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:12.932121 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4540054-5974-4ff9-bde0-838354fb6860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:12.932408 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4540054-5974-4ff9-bde0-838354fb6860 admin admin] PATH_INFO: `/v3/users/302fe3b65b8944219f6bc27690ab82e4` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:12.933522 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4540054-5974-4ff9-bde0-838354fb6860 admin admin] RBAC: Authorizing `identity:delete_user(user_id=302fe3b65b8944219f6bc27690ab82e4)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:10:12.935653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4540054-5974-4ff9-bde0-838354fb6860 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:10:13.103577 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e4540054-5974-4ff9-bde0-838354fb6860 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '302fe3b65b8944219f6bc27690ab82e4'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:10:13.104349 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e4540054-5974-4ff9-bde0-838354fb6860 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '302fe3b65b8944219f6bc27690ab82e4'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:10:13.112661 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e4540054-5974-4ff9-bde0-838354fb6860 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '302fe3b65b8944219f6bc27690ab82e4'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:10:13.126358 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e4540054-5974-4ff9-bde0-838354fb6860 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '302fe3b65b8944219f6bc27690ab82e4'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:10:13.127263 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e4540054-5974-4ff9-bde0-838354fb6860 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '302fe3b65b8944219f6bc27690ab82e4'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:10:13.190291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1225/2463] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:10:12 2022] DELETE /identity/v3/users/302fe3b65b8944219f6bc27690ab82e4 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:10:13.194694 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b9c13-ee69-42ad-b1df-7efaa6c86a18 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:13.195155 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b9c13-ee69-42ad-b1df-7efaa6c86a18 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:13.195282 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b9c13-ee69-42ad-b1df-7efaa6c86a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:13.214779 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c11b9c13-ee69-42ad-b1df-7efaa6c86a18 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:10:13.216031 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c11b9c13-ee69-42ad-b1df-7efaa6c86a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:10:13.284312 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1239/2464] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:10:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:10:13.293306 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4540054-5974-4ff9-bde0-838354fb6860 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:10:13.294742 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e4540054-5974-4ff9-bde0-838354fb6860 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:10:13.360303 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4540054-5974-4ff9-bde0-838354fb6860 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:10:13.364206 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc10c2-fe48-4096-839f-e1dc78853bc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:10:13.364848 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc10c2-fe48-4096-839f-e1dc78853bc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:13.365136 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc10c2-fe48-4096-839f-e1dc78853bc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:13.365472 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc10c2-fe48-4096-839f-e1dc78853bc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:13.375526 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc10c2-fe48-4096-839f-e1dc78853bc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:10:13.377076 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc10c2-fe48-4096-839f-e1dc78853bc4 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:10:13.386892 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1226/2465] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:10:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:10:13.530790 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bff768-bb37-47cd-9c48-6a278a790df8 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:13.531180 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bff768-bb37-47cd-9c48-6a278a790df8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:13.531356 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bff768-bb37-47cd-9c48-6a278a790df8 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:13.533166 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1240/2466] 10.222.0.48 () {54 vars in 921 bytes} [Fri Sep 9 21:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:10:13.539556 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92588efc-d067-4c8f-9dae-4822d5b1cb1f None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:13.540198 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92588efc-d067-4c8f-9dae-4822d5b1cb1f None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:13.540509 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92588efc-d067-4c8f-9dae-4822d5b1cb1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:13.563577 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-92588efc-d067-4c8f-9dae-4822d5b1cb1f None None] MFA Rules not processed for user `613b845a9b4d432daf8d478fc3970708`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:10:13.565306 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-92588efc-d067-4c8f-9dae-4822d5b1cb1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:10:13.589195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:10:13.654010 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1227/2467] 10.222.0.48 () {58 vars in 1018 bytes} [Fri Sep 9 21:10:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:10:13.666281 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bff768-bb37-47cd-9c48-6a278a790df8 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:10:13.668179 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-14bff768-bb37-47cd-9c48-6a278a790df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:10:13.728692 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bff768-bb37-47cd-9c48-6a278a790df8 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:10:13.733037 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b235fc-bcff-4c33-a00a-ee37ff92c3d1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7832e97d7e97439e821268d05816ae3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:10:13.733805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b235fc-bcff-4c33-a00a-ee37ff92c3d1 service glance] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:13.734090 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b235fc-bcff-4c33-a00a-ee37ff92c3d1 service glance] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:13.734430 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b235fc-bcff-4c33-a00a-ee37ff92c3d1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:13.752966 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b235fc-bcff-4c33-a00a-ee37ff92c3d1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:10:13.754427 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b235fc-bcff-4c33-a00a-ee37ff92c3d1 service glance] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:10:13.766519 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1241/2468] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:10:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:10:13.958954 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92588efc-d067-4c8f-9dae-4822d5b1cb1f None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:10:13.960611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-92588efc-d067-4c8f-9dae-4822d5b1cb1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:10:13.976221 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92588efc-d067-4c8f-9dae-4822d5b1cb1f None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:10:13.983672 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4263f2b-54f0-43b0-a310-b77be9a41649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:10:13.987231 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4263f2b-54f0-43b0-a310-b77be9a41649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:13.987624 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4263f2b-54f0-43b0-a310-b77be9a41649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:13.987972 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4263f2b-54f0-43b0-a310-b77be9a41649 admin admin] PATH_INFO: `/v3/projects/0ac874ab0dff440e88da50892a1d2089` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:13.989405 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4263f2b-54f0-43b0-a310-b77be9a41649 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ac874ab0dff440e88da50892a1d2089)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:10:13.991671 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4263f2b-54f0-43b0-a310-b77be9a41649 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:10:14.019409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba5826-03e5-4f8c-9fcf-429fa3c10f13 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:14.019804 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba5826-03e5-4f8c-9fcf-429fa3c10f13 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:14.020183 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba5826-03e5-4f8c-9fcf-429fa3c10f13 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:14.021899 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1242/2469] 10.222.0.48 () {54 vars in 921 bytes} [Fri Sep 9 21:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:10:14.027354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e9cb6-b6e0-4bb1-87cb-b8a8133fe138 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:14.027821 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e9cb6-b6e0-4bb1-87cb-b8a8133fe138 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:14.028208 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e9cb6-b6e0-4bb1-87cb-b8a8133fe138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:14.051350 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-bb2e9cb6-b6e0-4bb1-87cb-b8a8133fe138 None None] MFA Rules not processed for user `8f5dfac06ac9492a86503ba133c638a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:10:14.052737 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bb2e9cb6-b6e0-4bb1-87cb-b8a8133fe138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:10:14.059919 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1243/2470] 10.222.0.48 () {58 vars in 1018 bytes} [Fri Sep 9 21:10:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:10:14.077001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e9cb6-b6e0-4bb1-87cb-b8a8133fe138 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:10:14.077001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bb2e9cb6-b6e0-4bb1-87cb-b8a8133fe138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:10:14.085774 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e9cb6-b6e0-4bb1-87cb-b8a8133fe138 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:10:14.089728 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64882079-f56e-4723-9ff6-4a07d4d4ae0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:10:14.090425 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64882079-f56e-4723-9ff6-4a07d4d4ae0e service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:14.090758 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64882079-f56e-4723-9ff6-4a07d4d4ae0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:14.091094 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64882079-f56e-4723-9ff6-4a07d4d4ae0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:14.100439 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64882079-f56e-4723-9ff6-4a07d4d4ae0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:10:14.102045 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64882079-f56e-4723-9ff6-4a07d4d4ae0e service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:10:14.111149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1244/2471] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:10:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:10:14.114902 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e4263f2b-54f0-43b0-a310-b77be9a41649 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ac874ab0dff440e88da50892a1d2089', 'request_id': 'req-e4263f2b-54f0-43b0-a310-b77be9a41649'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:10:14.185950 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e4263f2b-54f0-43b0-a310-b77be9a41649 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ac874ab0dff440e88da50892a1d2089', 'request_id': 'req-e4263f2b-54f0-43b0-a310-b77be9a41649'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:10:14.193370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e4263f2b-54f0-43b0-a310-b77be9a41649 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ac874ab0dff440e88da50892a1d2089', 'request_id': 'req-e4263f2b-54f0-43b0-a310-b77be9a41649'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:10:14.195497 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d5b36-5707-41ea-a97f-37ff444940ad None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:14.196404 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d5b36-5707-41ea-a97f-37ff444940ad None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:14.196887 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d5b36-5707-41ea-a97f-37ff444940ad None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:14.198828 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1245/2472] 10.222.0.48 () {54 vars in 921 bytes} [Fri Sep 9 21:10:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:10:14.205526 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55274d83-a4f7-415c-97d7-fdb9c7f3b9ff None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:14.206059 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55274d83-a4f7-415c-97d7-fdb9c7f3b9ff None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:14.206436 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e4263f2b-54f0-43b0-a310-b77be9a41649 admin admin] The token cache is being invalidate because project 0ac874ab0dff440e88da50892a1d2089 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:10:14.206596 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55274d83-a4f7-415c-97d7-fdb9c7f3b9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:14.206743 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e4263f2b-54f0-43b0-a310-b77be9a41649 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:10:14.207939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e4263f2b-54f0-43b0-a310-b77be9a41649 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:10:14.209618 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1228/2473] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:10:13 2022] DELETE /identity/v3/projects/0ac874ab0dff440e88da50892a1d2089 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:10:14.214939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d24902-6bc4-49df-ae9a-203e6a8f382f None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:14.215230 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d24902-6bc4-49df-ae9a-203e6a8f382f None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:14.215510 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d24902-6bc4-49df-ae9a-203e6a8f382f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:14.230082 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-55274d83-a4f7-415c-97d7-fdb9c7f3b9ff None None] MFA Rules not processed for user `613b845a9b4d432daf8d478fc3970708`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:10:14.231444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-55274d83-a4f7-415c-97d7-fdb9c7f3b9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:10:14.248985 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-a5d24902-6bc4-49df-ae9a-203e6a8f382f None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:10:14.256214 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a5d24902-6bc4-49df-ae9a-203e6a8f382f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:10:14.259716 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:10:14.307224 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1246/2474] 10.222.0.48 () {58 vars in 1018 bytes} [Fri Sep 9 21:10:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:10:14.318010 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55274d83-a4f7-415c-97d7-fdb9c7f3b9ff None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:10:14.319347 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-55274d83-a4f7-415c-97d7-fdb9c7f3b9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:10:14.320318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1229/2475] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:10:14.327799 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7c5fc-2dbf-464f-b24c-e3d832b0d5f4 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:14.328117 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7c5fc-2dbf-464f-b24c-e3d832b0d5f4 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:14.328404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7c5fc-2dbf-464f-b24c-e3d832b0d5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:14.350168 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-d8e7c5fc-2dbf-464f-b24c-e3d832b0d5f4 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:10:14.351378 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d8e7c5fc-2dbf-464f-b24c-e3d832b0d5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:10:14.364105 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1230/2476] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:10:14.366993 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7c5fc-2dbf-464f-b24c-e3d832b0d5f4 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:10:14.378908 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7c5fc-2dbf-464f-b24c-e3d832b0d5f4 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:10:14.381730 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55274d83-a4f7-415c-97d7-fdb9c7f3b9ff None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:10:14.384522 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92dd24-4c33-48e9-a66e-a95c52bbca9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:10:14.385130 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92dd24-4c33-48e9-a66e-a95c52bbca9c admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:14.385409 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92dd24-4c33-48e9-a66e-a95c52bbca9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:14.385678 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92dd24-4c33-48e9-a66e-a95c52bbca9c admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:14.386329 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b92dd24-4c33-48e9-a66e-a95c52bbca9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:10:14.386629 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b92dd24-4c33-48e9-a66e-a95c52bbca9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:10:14.387519 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c3b8a-5798-4b0d-b166-d532700a8977 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:10:14.388157 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c3b8a-5798-4b0d-b166-d532700a8977 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:14.388280 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b92dd24-4c33-48e9-a66e-a95c52bbca9c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:10:14.388440 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c3b8a-5798-4b0d-b166-d532700a8977 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:14.388703 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c3b8a-5798-4b0d-b166-d532700a8977 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:14.400008 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c3b8a-5798-4b0d-b166-d532700a8977 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:10:14.401912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c3b8a-5798-4b0d-b166-d532700a8977 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:10:14.402038 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1231/2477] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:10:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:10:14.406823 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92dd24-4c33-48e9-a66e-a95c52bbca9c admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:10:14.414374 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1247/2478] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:10:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:10:14.419117 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92dd24-4c33-48e9-a66e-a95c52bbca9c admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:10:14.424665 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8940bc9-6bed-47e7-bffc-ea481b9ea0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:10:14.425344 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8940bc9-6bed-47e7-bffc-ea481b9ea0d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:14.425695 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8940bc9-6bed-47e7-bffc-ea481b9ea0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:14.426797 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8940bc9-6bed-47e7-bffc-ea481b9ea0d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:14.426797 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8940bc9-6bed-47e7-bffc-ea481b9ea0d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:10:14.429201 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8940bc9-6bed-47e7-bffc-ea481b9ea0d9 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:10:14.448632 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:10:14.520099 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1232/2479] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:10:14 2022] POST /identity/v3/projects => generated 389 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:10:14.521815 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c3b8a-5798-4b0d-b166-d532700a8977 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:10:14.532276 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c3b8a-5798-4b0d-b166-d532700a8977 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:10:14.604104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5629de34-27a2-4bfd-922e-a28e18dc68b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:10:14.609904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629de34-27a2-4bfd-922e-a28e18dc68b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:14.609904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629de34-27a2-4bfd-922e-a28e18dc68b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:14.610350 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629de34-27a2-4bfd-922e-a28e18dc68b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:14.611394 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5629de34-27a2-4bfd-922e-a28e18dc68b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:10:14.614158 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5629de34-27a2-4bfd-922e-a28e18dc68b0 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:10:14.712119 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1248/2480] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:10:14 2022] POST /identity/v3/users => generated 295 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:10:14.716342 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8940bc9-6bed-47e7-bffc-ea481b9ea0d9 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:10:14.727829 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8940bc9-6bed-47e7-bffc-ea481b9ea0d9 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:10:14.734201 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fea351-6a64-4177-8d89-61b3f783f925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:10:14.734947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fea351-6a64-4177-8d89-61b3f783f925 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:14.735226 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fea351-6a64-4177-8d89-61b3f783f925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:14.735497 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fea351-6a64-4177-8d89-61b3f783f925 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:14.736188 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fea351-6a64-4177-8d89-61b3f783f925 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:10:14.737790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fea351-6a64-4177-8d89-61b3f783f925 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:10:14.749125 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1233/2481] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:10:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:10:14.753379 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5629de34-27a2-4bfd-922e-a28e18dc68b0 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:10:14.765746 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5629de34-27a2-4bfd-922e-a28e18dc68b0 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:10:14.771755 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b764d-c582-4851-b2c4-0bbb0fb15e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:10:14.772980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b764d-c582-4851-b2c4-0bbb0fb15e47 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:14.773433 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b764d-c582-4851-b2c4-0bbb0fb15e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:14.773711 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b764d-c582-4851-b2c4-0bbb0fb15e47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:14.774414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b764d-c582-4851-b2c4-0bbb0fb15e47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:10:14.777205 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b764d-c582-4851-b2c4-0bbb0fb15e47 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:10:14.790683 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1249/2482] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:10:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:10:14.799385 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fea351-6a64-4177-8d89-61b3f783f925 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:10:14.810315 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fea351-6a64-4177-8d89-61b3f783f925 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:10:14.821280 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04afcf-0f17-482f-bdf6-4886510b38e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:10:14.821280 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04afcf-0f17-482f-bdf6-4886510b38e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:14.821280 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04afcf-0f17-482f-bdf6-4886510b38e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:14.821280 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04afcf-0f17-482f-bdf6-4886510b38e4 admin admin] PATH_INFO: `/v3/projects/ad384efe9192465c92f6161d765c416f/users/e766febe7ad24207b44e7702313bf63a/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:14.856890 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc04afcf-0f17-482f-bdf6-4886510b38e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad384efe9192465c92f6161d765c416f, user_id=e766febe7ad24207b44e7702313bf63a, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:10:14.859790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc04afcf-0f17-482f-bdf6-4886510b38e4 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:10:14.956586 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1234/2483] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:10:14 2022] PUT /identity/v3/projects/ad384efe9192465c92f6161d765c416f/users/e766febe7ad24207b44e7702313bf63a/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:10:14.961805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87d6c4-78f7-42eb-be34-a17df5481ef9 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:14.962062 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87d6c4-78f7-42eb-be34-a17df5481ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:14.962418 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87d6c4-78f7-42eb-be34-a17df5481ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:14.996644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ac87d6c4-78f7-42eb-be34-a17df5481ef9 None None] MFA Rules not processed for user `e766febe7ad24207b44e7702313bf63a`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:10:14.997926 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ac87d6c4-78f7-42eb-be34-a17df5481ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:10:15.051832 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:10:15.087586 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1250/2484] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:10:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:10:15.098457 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04afcf-0f17-482f-bdf6-4886510b38e4 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:10:15.101053 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-bc04afcf-0f17-482f-bdf6-4886510b38e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:10:15.160320 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04afcf-0f17-482f-bdf6-4886510b38e4 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:10:15.164189 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b4250-9cd6-4a9d-b27d-ee9e141fdc14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:10:15.164879 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b4250-9cd6-4a9d-b27d-ee9e141fdc14 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:15.165219 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b4250-9cd6-4a9d-b27d-ee9e141fdc14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:15.165554 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b4250-9cd6-4a9d-b27d-ee9e141fdc14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:15.174246 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b4250-9cd6-4a9d-b27d-ee9e141fdc14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:10:15.175748 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b4250-9cd6-4a9d-b27d-ee9e141fdc14 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:10:15.184913 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1235/2485] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:10:15 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:10:18.280575 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87d6c4-78f7-42eb-be34-a17df5481ef9 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:10:18.281993 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ac87d6c4-78f7-42eb-be34-a17df5481ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:10:18.347460 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87d6c4-78f7-42eb-be34-a17df5481ef9 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:10:18.351208 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5978702-9a1c-4ed2-8f72-2786497fc2d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:10:18.351825 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5978702-9a1c-4ed2-8f72-2786497fc2d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:18.352130 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5978702-9a1c-4ed2-8f72-2786497fc2d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:18.352412 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5978702-9a1c-4ed2-8f72-2786497fc2d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:18.355914 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a5978702-9a1c-4ed2-8f72-2786497fc2d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:10:18.446000 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5978702-9a1c-4ed2-8f72-2786497fc2d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:10:18.447491 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5978702-9a1c-4ed2-8f72-2786497fc2d0 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:10:18.460165 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1251/2486] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:10:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:10:19.097500 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b4250-9cd6-4a9d-b27d-ee9e141fdc14 service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:10:19.106726 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b4250-9cd6-4a9d-b27d-ee9e141fdc14 service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:10:19.110804 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b6ac5-f84b-4d41-9382-59a60cb010ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:10:19.111383 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b6ac5-f84b-4d41-9382-59a60cb010ce service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:19.111657 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b6ac5-f84b-4d41-9382-59a60cb010ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:19.111924 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b6ac5-f84b-4d41-9382-59a60cb010ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:19.113665 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-be9b6ac5-f84b-4d41-9382-59a60cb010ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:10:19.127140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b6ac5-f84b-4d41-9382-59a60cb010ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:10:19.128576 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b6ac5-f84b-4d41-9382-59a60cb010ce service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:10:19.139136 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1236/2487] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:10:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:10:28.454003 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=67 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:10:29.134440 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=49,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:10:30.364892 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203bba60-7a0d-4439-8ea7-9f2ceda33c32 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:30.366841 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203bba60-7a0d-4439-8ea7-9f2ceda33c32 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:30.366841 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203bba60-7a0d-4439-8ea7-9f2ceda33c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:30.387980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-203bba60-7a0d-4439-8ea7-9f2ceda33c32 None None] MFA Rules not processed for user `8c21ea7d3cbb494886e60a4bd4f39145`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:10:30.389228 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-203bba60-7a0d-4439-8ea7-9f2ceda33c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:10:30.457520 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1252/2488] 10.222.0.48 () {58 vars in 1009 bytes} [Fri Sep 9 21:10:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:10:30.473308 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b6ac5-f84b-4d41-9382-59a60cb010ce service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:10:30.474763 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-be9b6ac5-f84b-4d41-9382-59a60cb010ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:10:30.550081 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b6ac5-f84b-4d41-9382-59a60cb010ce service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:10:30.555500 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91b69b-3345-45c4-a31c-b9b3401d5967 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d28fa5d49d44120bd5eaec2d4e5fdbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:10:30.556146 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91b69b-3345-45c4-a31c-b9b3401d5967 service swift] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:30.556432 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91b69b-3345-45c4-a31c-b9b3401d5967 service swift] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:30.556701 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91b69b-3345-45c4-a31c-b9b3401d5967 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:30.566247 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b91b69b-3345-45c4-a31c-b9b3401d5967 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:10:30.568957 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b91b69b-3345-45c4-a31c-b9b3401d5967 service swift] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:10:30.579336 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1237/2489] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Sep 9 21:10:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 21:10:34.937994 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203bba60-7a0d-4439-8ea7-9f2ceda33c32 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:10:34.939547 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-203bba60-7a0d-4439-8ea7-9f2ceda33c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:10:34.956982 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203bba60-7a0d-4439-8ea7-9f2ceda33c32 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:10:34.964278 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cfcdb2-5711-40c3-ae22-944cae0840ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:10:34.964972 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cfcdb2-5711-40c3-ae22-944cae0840ec service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:34.965355 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cfcdb2-5711-40c3-ae22-944cae0840ec service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:34.965659 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cfcdb2-5711-40c3-ae22-944cae0840ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:34.968048 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-52cfcdb2-5711-40c3-ae22-944cae0840ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:10:35.037684 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cfcdb2-5711-40c3-ae22-944cae0840ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:10:35.039530 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cfcdb2-5711-40c3-ae22-944cae0840ec service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:10:35.050082 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1253/2490] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:10:34 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:10:40.574482 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=32 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:10:45.044857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=53 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:10:47.174989 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91b69b-3345-45c4-a31c-b9b3401d5967 service swift] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:10:47.238678 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91b69b-3345-45c4-a31c-b9b3401d5967 service swift] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:10:47.243321 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb7334-03eb-4250-9d9f-9b576f6b15d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:10:47.243869 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb7334-03eb-4250-9d9f-9b576f6b15d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:47.244179 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb7334-03eb-4250-9d9f-9b576f6b15d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:47.244482 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb7334-03eb-4250-9d9f-9b576f6b15d8 admin admin] PATH_INFO: `/v3/users/e766febe7ad24207b44e7702313bf63a` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:47.245567 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb7334-03eb-4250-9d9f-9b576f6b15d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e766febe7ad24207b44e7702313bf63a)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:10:47.247490 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb7334-03eb-4250-9d9f-9b576f6b15d8 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:10:47.417255 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-51fb7334-03eb-4250-9d9f-9b576f6b15d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e766febe7ad24207b44e7702313bf63a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:10:47.418091 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-51fb7334-03eb-4250-9d9f-9b576f6b15d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e766febe7ad24207b44e7702313bf63a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:10:47.426260 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-51fb7334-03eb-4250-9d9f-9b576f6b15d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e766febe7ad24207b44e7702313bf63a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:10:47.442563 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-51fb7334-03eb-4250-9d9f-9b576f6b15d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e766febe7ad24207b44e7702313bf63a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:10:47.443649 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-51fb7334-03eb-4250-9d9f-9b576f6b15d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e766febe7ad24207b44e7702313bf63a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:10:47.522196 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1238/2491] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:10:47 2022] DELETE /identity/v3/users/e766febe7ad24207b44e7702313bf63a => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:10:47.526753 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897f17d-831c-4349-927d-d577c3c11f44 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:47.527321 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897f17d-831c-4349-927d-d577c3c11f44 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:47.527321 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897f17d-831c-4349-927d-d577c3c11f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:47.547047 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9897f17d-831c-4349-927d-d577c3c11f44 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:10:47.548148 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9897f17d-831c-4349-927d-d577c3c11f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:10:47.609582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1254/2492] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:10:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:10:47.619353 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb7334-03eb-4250-9d9f-9b576f6b15d8 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:10:47.620752 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-51fb7334-03eb-4250-9d9f-9b576f6b15d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:10:47.679929 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb7334-03eb-4250-9d9f-9b576f6b15d8 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:10:47.683525 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0367a5eb-46ae-4b0d-bbe5-01466341fd3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:10:47.684153 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367a5eb-46ae-4b0d-bbe5-01466341fd3f service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:47.684432 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367a5eb-46ae-4b0d-bbe5-01466341fd3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:47.684702 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367a5eb-46ae-4b0d-bbe5-01466341fd3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:47.693480 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0367a5eb-46ae-4b0d-bbe5-01466341fd3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:10:47.694893 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0367a5eb-46ae-4b0d-bbe5-01466341fd3f service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:10:47.705056 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1239/2493] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:10:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:10:48.117698 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9897f17d-831c-4349-927d-d577c3c11f44 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:10:48.119119 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9897f17d-831c-4349-927d-d577c3c11f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:10:48.132941 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9897f17d-831c-4349-927d-d577c3c11f44 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:10:48.136122 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a77ac12-a78c-4d2b-b4de-d830631ec7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:10:48.136766 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77ac12-a78c-4d2b-b4de-d830631ec7b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:48.137119 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77ac12-a78c-4d2b-b4de-d830631ec7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:48.137394 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77ac12-a78c-4d2b-b4de-d830631ec7b7 admin admin] PATH_INFO: `/v3/projects/ad384efe9192465c92f6161d765c416f` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:48.138589 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a77ac12-a78c-4d2b-b4de-d830631ec7b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad384efe9192465c92f6161d765c416f)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:10:48.141105 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a77ac12-a78c-4d2b-b4de-d830631ec7b7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:10:48.231004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2a77ac12-a78c-4d2b-b4de-d830631ec7b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad384efe9192465c92f6161d765c416f', 'request_id': 'req-2a77ac12-a78c-4d2b-b4de-d830631ec7b7'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:10:48.237560 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2a77ac12-a78c-4d2b-b4de-d830631ec7b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad384efe9192465c92f6161d765c416f', 'request_id': 'req-2a77ac12-a78c-4d2b-b4de-d830631ec7b7'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:10:48.287425 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2a77ac12-a78c-4d2b-b4de-d830631ec7b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad384efe9192465c92f6161d765c416f', 'request_id': 'req-2a77ac12-a78c-4d2b-b4de-d830631ec7b7'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:10:48.298478 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2a77ac12-a78c-4d2b-b4de-d830631ec7b7 admin admin] The token cache is being invalidate because project ad384efe9192465c92f6161d765c416f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:10:48.298770 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2a77ac12-a78c-4d2b-b4de-d830631ec7b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:10:48.299402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2a77ac12-a78c-4d2b-b4de-d830631ec7b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:10:48.301036 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1255/2494] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:10:48 2022] DELETE /identity/v3/projects/ad384efe9192465c92f6161d765c416f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:10:49.949326 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0367a5eb-46ae-4b0d-bbe5-01466341fd3f service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:10:49.959431 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0367a5eb-46ae-4b0d-bbe5-01466341fd3f service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:10:49.979582 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:10:50.010016 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de529dc3-5f54-4005-9100-5ae3e26e6cbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:10:50.010759 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de529dc3-5f54-4005-9100-5ae3e26e6cbd service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:50.011046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de529dc3-5f54-4005-9100-5ae3e26e6cbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:50.011318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de529dc3-5f54-4005-9100-5ae3e26e6cbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:50.013123 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-de529dc3-5f54-4005-9100-5ae3e26e6cbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:10:50.023582 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de529dc3-5f54-4005-9100-5ae3e26e6cbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:10:50.025518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de529dc3-5f54-4005-9100-5ae3e26e6cbd service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:10:50.033481 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1240/2495] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:10:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:10:58.295340 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:00.104168 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=16 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:06.151395 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fc277-eb7e-419e-b3b7-3a8a831f978d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:06.151902 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fc277-eb7e-419e-b3b7-3a8a831f978d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:06.152025 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fc277-eb7e-419e-b3b7-3a8a831f978d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:06.174091 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-226fc277-eb7e-419e-b3b7-3a8a831f978d None None] MFA Rules not processed for user `8c21ea7d3cbb494886e60a4bd4f39145`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:06.175283 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-226fc277-eb7e-419e-b3b7-3a8a831f978d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:06.236950 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1256/2496] 10.222.0.48 () {58 vars in 967 bytes} [Fri Sep 9 21:11:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:11:06.259626 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de529dc3-5f54-4005-9100-5ae3e26e6cbd service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:06.261009 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-de529dc3-5f54-4005-9100-5ae3e26e6cbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:06.330114 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de529dc3-5f54-4005-9100-5ae3e26e6cbd service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:06.335176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52c5c1-4b73-4f8c-bfeb-4b624aed0991 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d28fa5d49d44120bd5eaec2d4e5fdbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:06.335783 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52c5c1-4b73-4f8c-bfeb-4b624aed0991 service swift] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:06.336062 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52c5c1-4b73-4f8c-bfeb-4b624aed0991 service swift] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:06.336364 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52c5c1-4b73-4f8c-bfeb-4b624aed0991 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:06.345017 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52c5c1-4b73-4f8c-bfeb-4b624aed0991 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:06.346589 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52c5c1-4b73-4f8c-bfeb-4b624aed0991 service swift] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:06.354574 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1241/2497] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Sep 9 21:11:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 21:11:16.229495 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=25 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:16.351405 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=32 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:30.600506 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226fc277-eb7e-419e-b3b7-3a8a831f978d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:30.601953 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-226fc277-eb7e-419e-b3b7-3a8a831f978d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:30.673872 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226fc277-eb7e-419e-b3b7-3a8a831f978d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:30.679367 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5502d4d-9e4d-4bba-9411-159b52183fab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:30.679976 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5502d4d-9e4d-4bba-9411-159b52183fab service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:30.680282 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5502d4d-9e4d-4bba-9411-159b52183fab service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:30.680553 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5502d4d-9e4d-4bba-9411-159b52183fab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:30.682292 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d5502d4d-9e4d-4bba-9411-159b52183fab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:30.695138 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5502d4d-9e4d-4bba-9411-159b52183fab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:30.696772 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5502d4d-9e4d-4bba-9411-159b52183fab service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:30.707720 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1257/2498] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:11:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:11:40.702777 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=34 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:42.270508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52c5c1-4b73-4f8c-bfeb-4b624aed0991 service swift] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:42.271957 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-dd52c5c1-4b73-4f8c-bfeb-4b624aed0991 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:42.287849 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52c5c1-4b73-4f8c-bfeb-4b624aed0991 service swift] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:42.293483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95fe645-38c5-44a3-8c48-e325b93e5e0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:42.295652 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95fe645-38c5-44a3-8c48-e325b93e5e0c service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:42.295652 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95fe645-38c5-44a3-8c48-e325b93e5e0c service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:42.295652 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95fe645-38c5-44a3-8c48-e325b93e5e0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:42.297099 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e95fe645-38c5-44a3-8c48-e325b93e5e0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:42.355175 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95fe645-38c5-44a3-8c48-e325b93e5e0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:42.356834 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95fe645-38c5-44a3-8c48-e325b93e5e0c service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:42.366644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1242/2499] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:11:42 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:11:52.362547 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=28 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:57.509625 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5502d4d-9e4d-4bba-9411-159b52183fab service nova] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:57.520850 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5502d4d-9e4d-4bba-9411-159b52183fab service nova] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:57.526360 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42757dc-01c3-4495-a21d-c7a1fc03777e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:57.526950 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42757dc-01c3-4495-a21d-c7a1fc03777e service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:57.527232 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42757dc-01c3-4495-a21d-c7a1fc03777e service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:57.527506 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42757dc-01c3-4495-a21d-c7a1fc03777e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:57.529238 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f42757dc-01c3-4495-a21d-c7a1fc03777e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:57.587349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42757dc-01c3-4495-a21d-c7a1fc03777e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:57.588975 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42757dc-01c3-4495-a21d-c7a1fc03777e service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:57.598633 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1258/2500] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:11:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:12:02.455420 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95fe645-38c5-44a3-8c48-e325b93e5e0c service nova] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:02.457340 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e95fe645-38c5-44a3-8c48-e325b93e5e0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:02.535833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95fe645-38c5-44a3-8c48-e325b93e5e0c service nova] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:02.542437 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19985bf-e09d-461c-bcba-99d2661ae0cb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4f6de2065d4fbea3fa141da760f708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:02.543096 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19985bf-e09d-461c-bcba-99d2661ae0cb service placement] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:02.543377 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19985bf-e09d-461c-bcba-99d2661ae0cb service placement] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:02.543643 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19985bf-e09d-461c-bcba-99d2661ae0cb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:02.545477 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b19985bf-e09d-461c-bcba-99d2661ae0cb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:02.558403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19985bf-e09d-461c-bcba-99d2661ae0cb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:02.560008 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19985bf-e09d-461c-bcba-99d2661ae0cb service placement] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:02.570768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1243/2501] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:12:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:12:02.589429 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42757dc-01c3-4495-a21d-c7a1fc03777e service nova] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:02.592230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f42757dc-01c3-4495-a21d-c7a1fc03777e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:02.664010 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42757dc-01c3-4495-a21d-c7a1fc03777e service nova] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:02.668804 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc82b184-07a2-45cd-9e82-bc6e1903ac97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:02.669445 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82b184-07a2-45cd-9e82-bc6e1903ac97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:02.669724 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82b184-07a2-45cd-9e82-bc6e1903ac97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:02.670003 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82b184-07a2-45cd-9e82-bc6e1903ac97 admin admin] PATH_INFO: `/v3/users/0d9129eaa2694ef6b3f006b67ef88c52` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:02.671284 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc82b184-07a2-45cd-9e82-bc6e1903ac97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d9129eaa2694ef6b3f006b67ef88c52)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:02.673274 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc82b184-07a2-45cd-9e82-bc6e1903ac97 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:02.899252 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-cc82b184-07a2-45cd-9e82-bc6e1903ac97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d9129eaa2694ef6b3f006b67ef88c52'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:12:02.899935 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-cc82b184-07a2-45cd-9e82-bc6e1903ac97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d9129eaa2694ef6b3f006b67ef88c52'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:12:02.900578 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-cc82b184-07a2-45cd-9e82-bc6e1903ac97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d9129eaa2694ef6b3f006b67ef88c52'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:12:02.907592 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-cc82b184-07a2-45cd-9e82-bc6e1903ac97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d9129eaa2694ef6b3f006b67ef88c52'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:12:02.920524 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-cc82b184-07a2-45cd-9e82-bc6e1903ac97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d9129eaa2694ef6b3f006b67ef88c52'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:12:02.970251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1259/2502] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:12:02 2022] DELETE /identity/v3/users/0d9129eaa2694ef6b3f006b67ef88c52 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:12:03.183950 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19985bf-e09d-461c-bcba-99d2661ae0cb service placement] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:03.196564 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19985bf-e09d-461c-bcba-99d2661ae0cb service placement] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:03.255235 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf7d68-6f3f-40c8-9077-c415ce4d9038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:03.255862 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf7d68-6f3f-40c8-9077-c415ce4d9038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:03.256174 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf7d68-6f3f-40c8-9077-c415ce4d9038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:03.256446 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf7d68-6f3f-40c8-9077-c415ce4d9038 admin admin] PATH_INFO: `/v3/projects/ce9d99ec2c254abaa341c4cab6f2c534` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:03.257593 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddf7d68-6f3f-40c8-9077-c415ce4d9038 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce9d99ec2c254abaa341c4cab6f2c534)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:03.259575 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddf7d68-6f3f-40c8-9077-c415ce4d9038 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:03.348552 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1ddf7d68-6f3f-40c8-9077-c415ce4d9038 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce9d99ec2c254abaa341c4cab6f2c534', 'request_id': 'req-1ddf7d68-6f3f-40c8-9077-c415ce4d9038'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:12:03.398980 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1ddf7d68-6f3f-40c8-9077-c415ce4d9038 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce9d99ec2c254abaa341c4cab6f2c534', 'request_id': 'req-1ddf7d68-6f3f-40c8-9077-c415ce4d9038'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:12:03.405539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1ddf7d68-6f3f-40c8-9077-c415ce4d9038 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce9d99ec2c254abaa341c4cab6f2c534', 'request_id': 'req-1ddf7d68-6f3f-40c8-9077-c415ce4d9038'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:12:03.416536 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1ddf7d68-6f3f-40c8-9077-c415ce4d9038 admin admin] The token cache is being invalidate because project ce9d99ec2c254abaa341c4cab6f2c534 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:12:03.416827 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1ddf7d68-6f3f-40c8-9077-c415ce4d9038 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:12:03.417463 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1ddf7d68-6f3f-40c8-9077-c415ce4d9038 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:12:03.419040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1244/2503] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:12:03 2022] DELETE /identity/v3/projects/ce9d99ec2c254abaa341c4cab6f2c534 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:12:08.485699 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a25c2-2e09-45ce-aaea-49532fd1cde6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:08.486305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a25c2-2e09-45ce-aaea-49532fd1cde6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:08.486305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a25c2-2e09-45ce-aaea-49532fd1cde6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:08.507276 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-8a4a25c2-2e09-45ce-aaea-49532fd1cde6 None None] MFA Rules not processed for user `c764d9778ef74b709cedbccabac37af0`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:08.509027 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8a4a25c2-2e09-45ce-aaea-49532fd1cde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:08.527633 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:08.575750 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1260/2504] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:12:08 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:12:13.413401 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:13.646022 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf7d68-6f3f-40c8-9077-c415ce4d9038 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:13.647452 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-1ddf7d68-6f3f-40c8-9077-c415ce4d9038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:13.728510 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf7d68-6f3f-40c8-9077-c415ce4d9038 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:13.734022 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c9e60-24fc-4dc0-b8f8-0e3a1af6a7ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:13.734636 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c9e60-24fc-4dc0-b8f8-0e3a1af6a7ee service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:13.734915 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c9e60-24fc-4dc0-b8f8-0e3a1af6a7ee service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:13.735185 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c9e60-24fc-4dc0-b8f8-0e3a1af6a7ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:13.737001 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-9b1c9e60-24fc-4dc0-b8f8-0e3a1af6a7ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:13.749069 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1c9e60-24fc-4dc0-b8f8-0e3a1af6a7ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:13.750680 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1c9e60-24fc-4dc0-b8f8-0e3a1af6a7ee service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:13.761455 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1245/2505] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:12:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:12:18.594018 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=17 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:23.756860 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=34 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:30.914271 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a25c2-2e09-45ce-aaea-49532fd1cde6 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:30.930157 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a25c2-2e09-45ce-aaea-49532fd1cde6 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:30.935767 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03954a48-81c1-4a95-9c3e-03c06816e81f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:30.936405 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03954a48-81c1-4a95-9c3e-03c06816e81f service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:30.936685 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03954a48-81c1-4a95-9c3e-03c06816e81f service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:30.936954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03954a48-81c1-4a95-9c3e-03c06816e81f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:30.938737 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-03954a48-81c1-4a95-9c3e-03c06816e81f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:31.000650 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03954a48-81c1-4a95-9c3e-03c06816e81f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:31.002955 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03954a48-81c1-4a95-9c3e-03c06816e81f service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:31.012535 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1261/2506] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:12:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:12:39.166325 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60c639-0632-44c5-bf03-618c3d52dab5 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:39.166795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60c639-0632-44c5-bf03-618c3d52dab5 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:39.168394 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60c639-0632-44c5-bf03-618c3d52dab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:39.186085 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-de60c639-0632-44c5-bf03-618c3d52dab5 None None] MFA Rules not processed for user `710e49db188845bbae5a183c6657e07f`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:39.191434 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-de60c639-0632-44c5-bf03-618c3d52dab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:39.242088 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1246/2507] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:12:39 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:12:39.539222 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03954a48-81c1-4a95-9c3e-03c06816e81f service nova] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:39.540678 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-03954a48-81c1-4a95-9c3e-03c06816e81f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:39.596650 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03954a48-81c1-4a95-9c3e-03c06816e81f service nova] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:39.600312 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8e83f-6cc9-4001-8914-482fc9ce806e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:39.600906 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8e83f-6cc9-4001-8914-482fc9ce806e service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:39.601182 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8e83f-6cc9-4001-8914-482fc9ce806e service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:39.601450 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8e83f-6cc9-4001-8914-482fc9ce806e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:39.603156 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e9a8e83f-6cc9-4001-8914-482fc9ce806e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:39.613250 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a8e83f-6cc9-4001-8914-482fc9ce806e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:39.614672 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a8e83f-6cc9-4001-8914-482fc9ce806e service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:39.622359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1262/2508] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:12:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:12:40.015180 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de60c639-0632-44c5-bf03-618c3d52dab5 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:40.016715 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-de60c639-0632-44c5-bf03-618c3d52dab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:40.088441 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de60c639-0632-44c5-bf03-618c3d52dab5 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:40.093481 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876f5f7-03ee-41bc-98f4-8a43529149af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4f6de2065d4fbea3fa141da760f708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:40.094083 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876f5f7-03ee-41bc-98f4-8a43529149af service placement] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:40.094365 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876f5f7-03ee-41bc-98f4-8a43529149af service placement] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:40.094640 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876f5f7-03ee-41bc-98f4-8a43529149af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:40.096910 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8876f5f7-03ee-41bc-98f4-8a43529149af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:40.108091 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876f5f7-03ee-41bc-98f4-8a43529149af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:40.109668 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876f5f7-03ee-41bc-98f4-8a43529149af service placement] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:40.119986 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1247/2509] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:12:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:12:49.459518 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8e83f-6cc9-4001-8914-482fc9ce806e service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:49.461040 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e9a8e83f-6cc9-4001-8914-482fc9ce806e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:49.477299 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8e83f-6cc9-4001-8914-482fc9ce806e service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:49.482961 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb4cc2-652f-4af8-a219-e27ed8724834 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:49.483588 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb4cc2-652f-4af8-a219-e27ed8724834 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:49.483865 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb4cc2-652f-4af8-a219-e27ed8724834 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:49.484171 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb4cc2-652f-4af8-a219-e27ed8724834 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:49.486351 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-dffb4cc2-652f-4af8-a219-e27ed8724834 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:49.498241 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffb4cc2-652f-4af8-a219-e27ed8724834 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:49.499820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffb4cc2-652f-4af8-a219-e27ed8724834 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:49.511118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1263/2510] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:12:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:12:50.115794 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=57 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:59.506507 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=60 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:17.126780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876f5f7-03ee-41bc-98f4-8a43529149af service placement] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:17.138826 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876f5f7-03ee-41bc-98f4-8a43529149af service placement] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:17.144683 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0210d99-087b-460d-ac92-8251dea6fe1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:17.145263 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0210d99-087b-460d-ac92-8251dea6fe1a service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:17.145539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0210d99-087b-460d-ac92-8251dea6fe1a service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:17.145806 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0210d99-087b-460d-ac92-8251dea6fe1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:17.147486 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f0210d99-087b-460d-ac92-8251dea6fe1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:17.203255 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0210d99-087b-460d-ac92-8251dea6fe1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:17.205065 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0210d99-087b-460d-ac92-8251dea6fe1a service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:17.214266 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1248/2511] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:13:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:13:24.424114 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e2336-7d47-4be3-8514-c90380742f1c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:24.424114 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e2336-7d47-4be3-8514-c90380742f1c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:24.424114 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e2336-7d47-4be3-8514-c90380742f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:24.461574 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-799e2336-7d47-4be3-8514-c90380742f1c None None] MFA Rules not processed for user `f289a1bdd79b4ba0bf9499edb7729c9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:24.462866 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-799e2336-7d47-4be3-8514-c90380742f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:24.488146 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1264/2512] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:13:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:13:27.210266 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=28 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:34.486092 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=15 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:39.385738 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ebd32-20f6-4c42-bd1f-608ce1883c5a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:39.386198 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ebd32-20f6-4c42-bd1f-608ce1883c5a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:39.386291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ebd32-20f6-4c42-bd1f-608ce1883c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:39.405995 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-022ebd32-20f6-4c42-bd1f-608ce1883c5a None None] MFA Rules not processed for user `e3ef96f764244e5893f16a74d4169edb`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:39.407207 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-022ebd32-20f6-4c42-bd1f-608ce1883c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:39.461767 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1249/2513] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:13:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:13:40.115577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e2336-7d47-4be3-8514-c90380742f1c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:40.117136 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-799e2336-7d47-4be3-8514-c90380742f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:40.175714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e2336-7d47-4be3-8514-c90380742f1c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:40.179743 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af86f79-91a5-4677-ac65-bf54235e1670 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:40.180397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af86f79-91a5-4677-ac65-bf54235e1670 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:40.180729 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af86f79-91a5-4677-ac65-bf54235e1670 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:40.181030 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af86f79-91a5-4677-ac65-bf54235e1670 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:40.182805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6af86f79-91a5-4677-ac65-bf54235e1670 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:40.288724 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af86f79-91a5-4677-ac65-bf54235e1670 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:40.290174 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af86f79-91a5-4677-ac65-bf54235e1670 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:40.311644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1265/2514] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:13:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:13:44.773995 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ebd32-20f6-4c42-bd1f-608ce1883c5a None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:44.775626 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-022ebd32-20f6-4c42-bd1f-608ce1883c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:44.819130 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ebd32-20f6-4c42-bd1f-608ce1883c5a None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:44.824590 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4b3d0-462f-4e28-9759-7ea4dcf570b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4f6de2065d4fbea3fa141da760f708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:44.825222 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4b3d0-462f-4e28-9759-7ea4dcf570b2 service placement] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:44.825494 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4b3d0-462f-4e28-9759-7ea4dcf570b2 service placement] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:44.825946 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4b3d0-462f-4e28-9759-7ea4dcf570b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:44.829637 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-fcc4b3d0-462f-4e28-9759-7ea4dcf570b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:44.841895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc4b3d0-462f-4e28-9759-7ea4dcf570b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:44.843590 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc4b3d0-462f-4e28-9759-7ea4dcf570b2 service placement] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:44.854801 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1250/2515] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:13:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:13:44.932228 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af86f79-91a5-4677-ac65-bf54235e1670 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:44.933623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6af86f79-91a5-4677-ac65-bf54235e1670 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:44.946405 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af86f79-91a5-4677-ac65-bf54235e1670 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:44.950996 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2306a-6f44-4601-b8a8-2d7ed78e1fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:44.951800 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2306a-6f44-4601-b8a8-2d7ed78e1fd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:44.952308 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2306a-6f44-4601-b8a8-2d7ed78e1fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:44.952586 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2306a-6f44-4601-b8a8-2d7ed78e1fd1 admin admin] PATH_INFO: `/v3/users/c764d9778ef74b709cedbccabac37af0` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:44.953700 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2306a-6f44-4601-b8a8-2d7ed78e1fd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c764d9778ef74b709cedbccabac37af0)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:44.956485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2306a-6f44-4601-b8a8-2d7ed78e1fd1 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:45.143169 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-52a2306a-6f44-4601-b8a8-2d7ed78e1fd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c764d9778ef74b709cedbccabac37af0'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:13:45.143837 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-52a2306a-6f44-4601-b8a8-2d7ed78e1fd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c764d9778ef74b709cedbccabac37af0'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:13:45.144470 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-52a2306a-6f44-4601-b8a8-2d7ed78e1fd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c764d9778ef74b709cedbccabac37af0'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:13:45.151780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-52a2306a-6f44-4601-b8a8-2d7ed78e1fd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c764d9778ef74b709cedbccabac37af0'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:13:45.165290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-52a2306a-6f44-4601-b8a8-2d7ed78e1fd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c764d9778ef74b709cedbccabac37af0'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:13:45.169130 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:45.281432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1266/2516] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:13:44 2022] DELETE /identity/v3/users/c764d9778ef74b709cedbccabac37af0 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:13:45.582421 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4b3d0-462f-4e28-9759-7ea4dcf570b2 service placement] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:45.599671 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4b3d0-462f-4e28-9759-7ea4dcf570b2 service placement] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:45.660507 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650eaef-acd3-4a6a-aac2-a4d29c442492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:45.661144 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650eaef-acd3-4a6a-aac2-a4d29c442492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:45.661421 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650eaef-acd3-4a6a-aac2-a4d29c442492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:45.661696 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650eaef-acd3-4a6a-aac2-a4d29c442492 admin admin] PATH_INFO: `/v3/projects/25acf91bdf524474b03d44271d08fba6` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:45.662816 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650eaef-acd3-4a6a-aac2-a4d29c442492 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25acf91bdf524474b03d44271d08fba6)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:45.664758 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650eaef-acd3-4a6a-aac2-a4d29c442492 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:45.771161 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8650eaef-acd3-4a6a-aac2-a4d29c442492 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25acf91bdf524474b03d44271d08fba6', 'request_id': 'req-8650eaef-acd3-4a6a-aac2-a4d29c442492'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:13:45.832476 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8650eaef-acd3-4a6a-aac2-a4d29c442492 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25acf91bdf524474b03d44271d08fba6', 'request_id': 'req-8650eaef-acd3-4a6a-aac2-a4d29c442492'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:13:45.839540 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8650eaef-acd3-4a6a-aac2-a4d29c442492 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25acf91bdf524474b03d44271d08fba6', 'request_id': 'req-8650eaef-acd3-4a6a-aac2-a4d29c442492'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:13:45.850974 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8650eaef-acd3-4a6a-aac2-a4d29c442492 admin admin] The token cache is being invalidate because project 25acf91bdf524474b03d44271d08fba6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:13:45.851301 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8650eaef-acd3-4a6a-aac2-a4d29c442492 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:13:45.851969 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8650eaef-acd3-4a6a-aac2-a4d29c442492 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:13:45.853542 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1251/2517] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:13:45 2022] DELETE /identity/v3/projects/25acf91bdf524474b03d44271d08fba6 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:13:45.858436 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b046577-7f9d-49e6-a148-0b57baae4616 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:45.858741 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b046577-7f9d-49e6-a148-0b57baae4616 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:45.859021 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b046577-7f9d-49e6-a148-0b57baae4616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:45.918125 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9b046577-7f9d-49e6-a148-0b57baae4616 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:45.919342 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9b046577-7f9d-49e6-a148-0b57baae4616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:45.979179 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1267/2518] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:13:45.987496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca4d77-06d6-4e0a-bd84-00f70f6c9af9 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:45.987857 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca4d77-06d6-4e0a-bd84-00f70f6c9af9 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:45.988174 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca4d77-06d6-4e0a-bd84-00f70f6c9af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:45.996133 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:46.009248 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-b4ca4d77-06d6-4e0a-bd84-00f70f6c9af9 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:46.010640 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b4ca4d77-06d6-4e0a-bd84-00f70f6c9af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:46.019112 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1252/2519] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:13:46.023351 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b046577-7f9d-49e6-a148-0b57baae4616 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:46.033723 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b046577-7f9d-49e6-a148-0b57baae4616 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:46.038385 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaff515-dadc-419f-ba73-d57516b7c491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:46.039020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaff515-dadc-419f-ba73-d57516b7c491 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:46.039301 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaff515-dadc-419f-ba73-d57516b7c491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:46.039576 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaff515-dadc-419f-ba73-d57516b7c491 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:46.040266 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaff515-dadc-419f-ba73-d57516b7c491 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:46.040576 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaff515-dadc-419f-ba73-d57516b7c491 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:46.042184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaff515-dadc-419f-ba73-d57516b7c491 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:46.054436 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1268/2520] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:46.058934 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca4d77-06d6-4e0a-bd84-00f70f6c9af9 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:46.068721 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca4d77-06d6-4e0a-bd84-00f70f6c9af9 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:46.073392 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a40fa-cce7-432f-a3ad-ce374209b35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:46.073978 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a40fa-cce7-432f-a3ad-ce374209b35b admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:46.074257 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a40fa-cce7-432f-a3ad-ce374209b35b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:46.074527 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a40fa-cce7-432f-a3ad-ce374209b35b admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:46.075152 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a40fa-cce7-432f-a3ad-ce374209b35b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:46.077082 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a40fa-cce7-432f-a3ad-ce374209b35b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:46.159328 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1253/2521] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:13:46 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:13:46.163463 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaff515-dadc-419f-ba73-d57516b7c491 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:46.174093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaff515-dadc-419f-ba73-d57516b7c491 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:46.231654 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18d671-a48e-49ab-b7ce-f6762070ca8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:46.232205 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18d671-a48e-49ab-b7ce-f6762070ca8d admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:46.232489 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18d671-a48e-49ab-b7ce-f6762070ca8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:46.232762 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18d671-a48e-49ab-b7ce-f6762070ca8d admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:46.233409 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f18d671-a48e-49ab-b7ce-f6762070ca8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:46.235317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f18d671-a48e-49ab-b7ce-f6762070ca8d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:46.289183 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1269/2522] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:13:46 2022] POST /identity/v3/users => generated 290 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:13:46.293373 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a40fa-cce7-432f-a3ad-ce374209b35b admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:46.304374 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a40fa-cce7-432f-a3ad-ce374209b35b admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:46.309064 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ba605-0318-4084-8ed9-961c7bde485c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:46.309652 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ba605-0318-4084-8ed9-961c7bde485c admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:46.309931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ba605-0318-4084-8ed9-961c7bde485c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:46.310232 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ba605-0318-4084-8ed9-961c7bde485c admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:46.310872 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6ba605-0318-4084-8ed9-961c7bde485c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:46.312485 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6ba605-0318-4084-8ed9-961c7bde485c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:46.322257 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1254/2523] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:13:46.326158 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18d671-a48e-49ab-b7ce-f6762070ca8d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:46.336796 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18d671-a48e-49ab-b7ce-f6762070ca8d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:46.341340 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ed967-bf9c-448e-94b7-3f65b0652055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:46.341922 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ed967-bf9c-448e-94b7-3f65b0652055 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:46.342201 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ed967-bf9c-448e-94b7-3f65b0652055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:46.342478 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ed967-bf9c-448e-94b7-3f65b0652055 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:46.343118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ed967-bf9c-448e-94b7-3f65b0652055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:46.344954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ed967-bf9c-448e-94b7-3f65b0652055 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:46.357026 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1270/2524] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:13:46.360953 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ba605-0318-4084-8ed9-961c7bde485c admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:46.370793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ba605-0318-4084-8ed9-961c7bde485c admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:46.375337 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0327f2c6-d831-4ba8-933d-9e4bde5bb390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:46.375977 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327f2c6-d831-4ba8-933d-9e4bde5bb390 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:46.376282 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327f2c6-d831-4ba8-933d-9e4bde5bb390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:46.376562 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327f2c6-d831-4ba8-933d-9e4bde5bb390 admin admin] PATH_INFO: `/v3/projects/c46605e6f99545ffad18ab7d560972c4/users/a8aafcc83f694fa39b28dcd3f16d6126/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:46.391856 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0327f2c6-d831-4ba8-933d-9e4bde5bb390 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c46605e6f99545ffad18ab7d560972c4, user_id=a8aafcc83f694fa39b28dcd3f16d6126, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:46.395112 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0327f2c6-d831-4ba8-933d-9e4bde5bb390 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:46.529864 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1255/2525] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:13:46 2022] PUT /identity/v3/projects/c46605e6f99545ffad18ab7d560972c4/users/a8aafcc83f694fa39b28dcd3f16d6126/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:13:46.540892 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cacbb27-9488-48b3-aed9-3b7df9f7a58e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:46.541251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cacbb27-9488-48b3-aed9-3b7df9f7a58e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:46.541597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cacbb27-9488-48b3-aed9-3b7df9f7a58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:46.569558 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5cacbb27-9488-48b3-aed9-3b7df9f7a58e None None] MFA Rules not processed for user `a8aafcc83f694fa39b28dcd3f16d6126`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:46.570915 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5cacbb27-9488-48b3-aed9-3b7df9f7a58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:46.590960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:46.638204 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1271/2526] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:13:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:13:46.644955 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0327f2c6-d831-4ba8-933d-9e4bde5bb390 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:46.824890 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0327f2c6-d831-4ba8-933d-9e4bde5bb390 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:46.829676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc20673c-fd1a-4856-92f3-197ae1e5b3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:46.830268 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20673c-fd1a-4856-92f3-197ae1e5b3cc admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:46.830542 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20673c-fd1a-4856-92f3-197ae1e5b3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:46.830806 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20673c-fd1a-4856-92f3-197ae1e5b3cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:46.831982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc20673c-fd1a-4856-92f3-197ae1e5b3cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:46.833884 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc20673c-fd1a-4856-92f3-197ae1e5b3cc admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:46.912592 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1256/2527] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:13:46 2022] POST /identity/v3/projects => generated 381 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:13:46.916695 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cacbb27-9488-48b3-aed9-3b7df9f7a58e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:46.927237 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cacbb27-9488-48b3-aed9-3b7df9f7a58e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:46.984127 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700eec4-5cc5-450e-94a1-528740a05910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:46.984817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700eec4-5cc5-450e-94a1-528740a05910 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:46.985097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700eec4-5cc5-450e-94a1-528740a05910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:46.985366 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700eec4-5cc5-450e-94a1-528740a05910 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:46.985995 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1700eec4-5cc5-450e-94a1-528740a05910 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:46.987930 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1700eec4-5cc5-450e-94a1-528740a05910 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:47.072894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1272/2528] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:13:46 2022] POST /identity/v3/users => generated 297 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:13:47.076951 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc20673c-fd1a-4856-92f3-197ae1e5b3cc admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:47.088137 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc20673c-fd1a-4856-92f3-197ae1e5b3cc admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:47.092687 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc241931-2117-4470-aba8-b42672c5ffcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:47.093273 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc241931-2117-4470-aba8-b42672c5ffcb admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:47.093545 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc241931-2117-4470-aba8-b42672c5ffcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:47.093812 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc241931-2117-4470-aba8-b42672c5ffcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:47.094447 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc241931-2117-4470-aba8-b42672c5ffcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:47.096166 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc241931-2117-4470-aba8-b42672c5ffcb admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:47.106249 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1257/2529] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:13:47.110179 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700eec4-5cc5-450e-94a1-528740a05910 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:47.120456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700eec4-5cc5-450e-94a1-528740a05910 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:47.124910 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3ffd7-674c-499e-a2cf-41aaf23cb20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:47.125549 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3ffd7-674c-499e-a2cf-41aaf23cb20d admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:47.125817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3ffd7-674c-499e-a2cf-41aaf23cb20d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:47.126087 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3ffd7-674c-499e-a2cf-41aaf23cb20d admin admin] PATH_INFO: `/v3/projects/63c55d91e8964395ab20771777600433/users/1def7db6787d437eb6dd663375e1e5e0/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:47.140894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce3ffd7-674c-499e-a2cf-41aaf23cb20d admin admin] RBAC: Authorizing `identity:create_grant(project_id=63c55d91e8964395ab20771777600433, user_id=1def7db6787d437eb6dd663375e1e5e0, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:47.143520 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce3ffd7-674c-499e-a2cf-41aaf23cb20d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:47.231452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1273/2530] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:13:47 2022] PUT /identity/v3/projects/63c55d91e8964395ab20771777600433/users/1def7db6787d437eb6dd663375e1e5e0/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:13:47.235378 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc241931-2117-4470-aba8-b42672c5ffcb admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:47.246697 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc241931-2117-4470-aba8-b42672c5ffcb admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:47.301887 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da6d2a-567a-40be-9e6c-7004bc4e809e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:47.302477 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da6d2a-567a-40be-9e6c-7004bc4e809e admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:47.302753 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da6d2a-567a-40be-9e6c-7004bc4e809e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:47.303016 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da6d2a-567a-40be-9e6c-7004bc4e809e admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:47.303636 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da6d2a-567a-40be-9e6c-7004bc4e809e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:47.305216 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da6d2a-567a-40be-9e6c-7004bc4e809e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:47.314416 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1258/2531] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:13:47.319135 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3ffd7-674c-499e-a2cf-41aaf23cb20d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:47.329611 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3ffd7-674c-499e-a2cf-41aaf23cb20d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:47.334153 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a5851-64db-4b0e-9334-c3b62a3008a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:47.334792 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a5851-64db-4b0e-9334-c3b62a3008a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:47.335074 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a5851-64db-4b0e-9334-c3b62a3008a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:47.335346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a5851-64db-4b0e-9334-c3b62a3008a3 admin admin] PATH_INFO: `/v3/projects/63c55d91e8964395ab20771777600433/users/1def7db6787d437eb6dd663375e1e5e0/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:47.337274 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a5851-64db-4b0e-9334-c3b62a3008a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63c55d91e8964395ab20771777600433, user_id=1def7db6787d437eb6dd663375e1e5e0, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:47.339888 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a5851-64db-4b0e-9334-c3b62a3008a3 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:47.349900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1274/2532] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:13:47 2022] PUT /identity/v3/projects/63c55d91e8964395ab20771777600433/users/1def7db6787d437eb6dd663375e1e5e0/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:13:47.361120 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63cd39a-d71d-4a26-8377-35d16f8574e6 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:47.361401 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63cd39a-d71d-4a26-8377-35d16f8574e6 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:47.361671 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63cd39a-d71d-4a26-8377-35d16f8574e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:47.381730 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:47.390014 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-d63cd39a-d71d-4a26-8377-35d16f8574e6 None None] MFA Rules not processed for user `1def7db6787d437eb6dd663375e1e5e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:47.394177 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d63cd39a-d71d-4a26-8377-35d16f8574e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:47.456982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1259/2533] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:13:47 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:13:47.468392 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a5851-64db-4b0e-9334-c3b62a3008a3 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:47.470028 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5b5a5851-64db-4b0e-9334-c3b62a3008a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:47.547793 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a5851-64db-4b0e-9334-c3b62a3008a3 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:47.551975 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb574657-73d3-4d99-a081-faa871989345 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:47.552598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb574657-73d3-4d99-a081-faa871989345 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:47.552874 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb574657-73d3-4d99-a081-faa871989345 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:47.553144 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb574657-73d3-4d99-a081-faa871989345 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:47.559862 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:47.562099 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb574657-73d3-4d99-a081-faa871989345 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:47.563520 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb574657-73d3-4d99-a081-faa871989345 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:47.572760 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1275/2534] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:13:47 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:13:47.749409 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63cd39a-d71d-4a26-8377-35d16f8574e6 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:47.759069 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63cd39a-d71d-4a26-8377-35d16f8574e6 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:47.762737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e1b18-1385-455a-ac45-679fd1a68252 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:47.763367 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e1b18-1385-455a-ac45-679fd1a68252 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:47.763648 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e1b18-1385-455a-ac45-679fd1a68252 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:47.763920 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e1b18-1385-455a-ac45-679fd1a68252 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:47.823902 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1e1b18-1385-455a-ac45-679fd1a68252 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:47.825356 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1e1b18-1385-455a-ac45-679fd1a68252 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:47.834861 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1260/2535] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:13:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:13:57.608686 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:57.830352 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=58 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:01.231526 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb574657-73d3-4d99-a081-faa871989345 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:01.296753 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb574657-73d3-4d99-a081-faa871989345 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:01.301602 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a42501-edc2-4bf6-86a1-575ebe527ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:01.302260 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a42501-edc2-4bf6-86a1-575ebe527ce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:01.302538 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a42501-edc2-4bf6-86a1-575ebe527ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:01.302809 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a42501-edc2-4bf6-86a1-575ebe527ce1 admin admin] PATH_INFO: `/v3/users/a8aafcc83f694fa39b28dcd3f16d6126` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:01.303940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a42501-edc2-4bf6-86a1-575ebe527ce1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8aafcc83f694fa39b28dcd3f16d6126)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:01.305916 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a42501-edc2-4bf6-86a1-575ebe527ce1 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:01.503917 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f3a42501-edc2-4bf6-86a1-575ebe527ce1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8aafcc83f694fa39b28dcd3f16d6126'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:01.504749 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f3a42501-edc2-4bf6-86a1-575ebe527ce1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8aafcc83f694fa39b28dcd3f16d6126'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:01.505449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f3a42501-edc2-4bf6-86a1-575ebe527ce1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8aafcc83f694fa39b28dcd3f16d6126'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:01.513041 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f3a42501-edc2-4bf6-86a1-575ebe527ce1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8aafcc83f694fa39b28dcd3f16d6126'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:01.526143 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f3a42501-edc2-4bf6-86a1-575ebe527ce1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8aafcc83f694fa39b28dcd3f16d6126'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:01.584720 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1276/2536] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:14:01 2022] DELETE /identity/v3/users/a8aafcc83f694fa39b28dcd3f16d6126 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:14:01.586496 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92a396-7d65-41da-a749-5bfea271b9c4 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:01.586947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92a396-7d65-41da-a749-5bfea271b9c4 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:01.587192 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92a396-7d65-41da-a749-5bfea271b9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:01.606762 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-2c92a396-7d65-41da-a749-5bfea271b9c4 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:01.607968 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2c92a396-7d65-41da-a749-5bfea271b9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:01.667396 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1261/2537] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:14:01.677405 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a42501-edc2-4bf6-86a1-575ebe527ce1 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:01.678939 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f3a42501-edc2-4bf6-86a1-575ebe527ce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:01.740143 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a42501-edc2-4bf6-86a1-575ebe527ce1 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:01.743836 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e082f-12c4-4c9f-bb2c-2278b9be590a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:01.744470 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e082f-12c4-4c9f-bb2c-2278b9be590a service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:01.744769 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e082f-12c4-4c9f-bb2c-2278b9be590a service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:01.745037 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e082f-12c4-4c9f-bb2c-2278b9be590a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:01.754820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e082f-12c4-4c9f-bb2c-2278b9be590a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:01.756888 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e082f-12c4-4c9f-bb2c-2278b9be590a service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:01.766991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1277/2538] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:14:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:14:02.164361 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c92a396-7d65-41da-a749-5bfea271b9c4 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:02.173138 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2c92a396-7d65-41da-a749-5bfea271b9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:02.178368 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c92a396-7d65-41da-a749-5bfea271b9c4 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:02.182980 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791afc0b-c776-4062-b7fa-20b5fb2b6967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:02.183642 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791afc0b-c776-4062-b7fa-20b5fb2b6967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:02.183912 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791afc0b-c776-4062-b7fa-20b5fb2b6967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:02.184263 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791afc0b-c776-4062-b7fa-20b5fb2b6967 admin admin] PATH_INFO: `/v3/projects/c46605e6f99545ffad18ab7d560972c4` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:02.185522 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791afc0b-c776-4062-b7fa-20b5fb2b6967 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c46605e6f99545ffad18ab7d560972c4)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:02.187413 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791afc0b-c776-4062-b7fa-20b5fb2b6967 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:02.288693 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-791afc0b-c776-4062-b7fa-20b5fb2b6967 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c46605e6f99545ffad18ab7d560972c4', 'request_id': 'req-791afc0b-c776-4062-b7fa-20b5fb2b6967'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:02.339829 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-791afc0b-c776-4062-b7fa-20b5fb2b6967 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c46605e6f99545ffad18ab7d560972c4', 'request_id': 'req-791afc0b-c776-4062-b7fa-20b5fb2b6967'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:02.346852 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-791afc0b-c776-4062-b7fa-20b5fb2b6967 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c46605e6f99545ffad18ab7d560972c4', 'request_id': 'req-791afc0b-c776-4062-b7fa-20b5fb2b6967'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:02.358328 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-791afc0b-c776-4062-b7fa-20b5fb2b6967 admin admin] The token cache is being invalidate because project c46605e6f99545ffad18ab7d560972c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:14:02.358611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-791afc0b-c776-4062-b7fa-20b5fb2b6967 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:02.359236 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-791afc0b-c776-4062-b7fa-20b5fb2b6967 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:02.360744 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1262/2539] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:14:02 2022] DELETE /identity/v3/projects/c46605e6f99545ffad18ab7d560972c4 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:14:02.365337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e082f-12c4-4c9f-bb2c-2278b9be590a service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:02.374798 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e082f-12c4-4c9f-bb2c-2278b9be590a service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:02.393110 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:02.431604 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023fbc3b-0e7c-4668-b851-e175683e73e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:02.432267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023fbc3b-0e7c-4668-b851-e175683e73e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:02.432570 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023fbc3b-0e7c-4668-b851-e175683e73e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:02.432867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023fbc3b-0e7c-4668-b851-e175683e73e7 admin admin] PATH_INFO: `/v3/users/1def7db6787d437eb6dd663375e1e5e0` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:02.433984 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023fbc3b-0e7c-4668-b851-e175683e73e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1def7db6787d437eb6dd663375e1e5e0)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:02.436024 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023fbc3b-0e7c-4668-b851-e175683e73e7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:02.604874 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-023fbc3b-0e7c-4668-b851-e175683e73e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1def7db6787d437eb6dd663375e1e5e0'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:02.605545 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-023fbc3b-0e7c-4668-b851-e175683e73e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1def7db6787d437eb6dd663375e1e5e0'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:02.606176 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-023fbc3b-0e7c-4668-b851-e175683e73e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1def7db6787d437eb6dd663375e1e5e0'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:02.613230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-023fbc3b-0e7c-4668-b851-e175683e73e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1def7db6787d437eb6dd663375e1e5e0'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:02.626096 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-023fbc3b-0e7c-4668-b851-e175683e73e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1def7db6787d437eb6dd663375e1e5e0'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:02.699480 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1278/2540] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:14:02 2022] DELETE /identity/v3/users/1def7db6787d437eb6dd663375e1e5e0 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:14:03.096261 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791afc0b-c776-4062-b7fa-20b5fb2b6967 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:03.097984 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-791afc0b-c776-4062-b7fa-20b5fb2b6967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:03.113877 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791afc0b-c776-4062-b7fa-20b5fb2b6967 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:03.170309 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea2ea2-9825-47e3-b6e2-c3f05948ed73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:03.170931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea2ea2-9825-47e3-b6e2-c3f05948ed73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:03.171211 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea2ea2-9825-47e3-b6e2-c3f05948ed73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:03.171483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea2ea2-9825-47e3-b6e2-c3f05948ed73 admin admin] PATH_INFO: `/v3/projects/63c55d91e8964395ab20771777600433` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:03.172610 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea2ea2-9825-47e3-b6e2-c3f05948ed73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63c55d91e8964395ab20771777600433)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:03.174483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea2ea2-9825-47e3-b6e2-c3f05948ed73 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:03.295405 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:03.311062 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-72ea2ea2-9825-47e3-b6e2-c3f05948ed73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63c55d91e8964395ab20771777600433', 'request_id': 'req-72ea2ea2-9825-47e3-b6e2-c3f05948ed73'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:03.420147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-72ea2ea2-9825-47e3-b6e2-c3f05948ed73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63c55d91e8964395ab20771777600433', 'request_id': 'req-72ea2ea2-9825-47e3-b6e2-c3f05948ed73'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:03.426976 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-72ea2ea2-9825-47e3-b6e2-c3f05948ed73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63c55d91e8964395ab20771777600433', 'request_id': 'req-72ea2ea2-9825-47e3-b6e2-c3f05948ed73'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:03.438025 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-72ea2ea2-9825-47e3-b6e2-c3f05948ed73 admin admin] The token cache is being invalidate because project 63c55d91e8964395ab20771777600433 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:14:03.438311 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-72ea2ea2-9825-47e3-b6e2-c3f05948ed73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:03.438949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-72ea2ea2-9825-47e3-b6e2-c3f05948ed73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:03.440426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1263/2541] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:14:03 2022] DELETE /identity/v3/projects/63c55d91e8964395ab20771777600433 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:14:03.445328 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de943c70-5c2c-4ca4-b734-f41f5e55866d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:03.445819 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de943c70-5c2c-4ca4-b734-f41f5e55866d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:03.446002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de943c70-5c2c-4ca4-b734-f41f5e55866d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:03.466891 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-de943c70-5c2c-4ca4-b734-f41f5e55866d None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:03.468117 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-de943c70-5c2c-4ca4-b734-f41f5e55866d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:03.527536 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1279/2542] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:14:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:14:03.535499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb43d9-e2f9-4ae2-8fd7-9595ab097df1 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:03.535791 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb43d9-e2f9-4ae2-8fd7-9595ab097df1 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:03.536088 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb43d9-e2f9-4ae2-8fd7-9595ab097df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:03.556255 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-03fb43d9-e2f9-4ae2-8fd7-9595ab097df1 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:03.557387 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-03fb43d9-e2f9-4ae2-8fd7-9595ab097df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:03.565191 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1264/2543] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:14:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:14:03.569525 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de943c70-5c2c-4ca4-b734-f41f5e55866d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:03.579189 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de943c70-5c2c-4ca4-b734-f41f5e55866d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:03.584467 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8256f-39d3-4ebc-9bd2-a64fa3daaf35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:03.585059 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8256f-39d3-4ebc-9bd2-a64fa3daaf35 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:03.585334 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8256f-39d3-4ebc-9bd2-a64fa3daaf35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:03.585605 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8256f-39d3-4ebc-9bd2-a64fa3daaf35 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:03.586275 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8256f-39d3-4ebc-9bd2-a64fa3daaf35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:14:03.586580 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8256f-39d3-4ebc-9bd2-a64fa3daaf35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:03.588193 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8256f-39d3-4ebc-9bd2-a64fa3daaf35 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:03.600006 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1280/2544] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:14:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:14:03.604095 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb43d9-e2f9-4ae2-8fd7-9595ab097df1 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:03.613027 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb43d9-e2f9-4ae2-8fd7-9595ab097df1 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:03.617588 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efeef2e-fb10-495f-a8ce-2afc221f5b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:03.618167 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efeef2e-fb10-495f-a8ce-2afc221f5b8d admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:03.618445 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efeef2e-fb10-495f-a8ce-2afc221f5b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:03.618712 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efeef2e-fb10-495f-a8ce-2afc221f5b8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:03.619342 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efeef2e-fb10-495f-a8ce-2afc221f5b8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:03.621243 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efeef2e-fb10-495f-a8ce-2afc221f5b8d admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:03.695935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1265/2545] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:14:03 2022] POST /identity/v3/projects => generated 365 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:14:03.700022 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8256f-39d3-4ebc-9bd2-a64fa3daaf35 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:03.710925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8256f-39d3-4ebc-9bd2-a64fa3daaf35 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:03.736426 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:03.765825 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1715747b-3c2d-492d-9d25-80d49f0f3e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:03.766426 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715747b-3c2d-492d-9d25-80d49f0f3e35 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:03.766704 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715747b-3c2d-492d-9d25-80d49f0f3e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:03.766982 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715747b-3c2d-492d-9d25-80d49f0f3e35 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:03.767612 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1715747b-3c2d-492d-9d25-80d49f0f3e35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:03.769568 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1715747b-3c2d-492d-9d25-80d49f0f3e35 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:03.865463 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1281/2546] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:14:03 2022] POST /identity/v3/users => generated 283 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:14:03.869526 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efeef2e-fb10-495f-a8ce-2afc221f5b8d admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:03.880008 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efeef2e-fb10-495f-a8ce-2afc221f5b8d admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:03.885219 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e8d011-0ea6-46c4-bc7a-161f90906e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:03.885838 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8d011-0ea6-46c4-bc7a-161f90906e35 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:03.886119 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8d011-0ea6-46c4-bc7a-161f90906e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:03.886391 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8d011-0ea6-46c4-bc7a-161f90906e35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:03.887035 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e8d011-0ea6-46c4-bc7a-161f90906e35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:03.888637 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e8d011-0ea6-46c4-bc7a-161f90906e35 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:03.898891 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1266/2547] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:14:03.902846 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1715747b-3c2d-492d-9d25-80d49f0f3e35 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:03.912661 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1715747b-3c2d-492d-9d25-80d49f0f3e35 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:03.917288 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9d3054-75e1-4bd2-9247-0d8fa2ab865d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:03.917870 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d3054-75e1-4bd2-9247-0d8fa2ab865d admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:03.918148 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d3054-75e1-4bd2-9247-0d8fa2ab865d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:03.918436 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d3054-75e1-4bd2-9247-0d8fa2ab865d admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:03.919051 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9d3054-75e1-4bd2-9247-0d8fa2ab865d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:03.920713 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9d3054-75e1-4bd2-9247-0d8fa2ab865d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:03.931050 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1282/2548] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:14:03.934939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e8d011-0ea6-46c4-bc7a-161f90906e35 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:03.945287 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e8d011-0ea6-46c4-bc7a-161f90906e35 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:03.949892 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d2a2a-ebb4-4444-9a10-330f9bc1e56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:03.950522 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d2a2a-ebb4-4444-9a10-330f9bc1e56d admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:03.950797 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d2a2a-ebb4-4444-9a10-330f9bc1e56d admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:03.951066 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d2a2a-ebb4-4444-9a10-330f9bc1e56d admin admin] PATH_INFO: `/v3/projects/9ed9f8de7f9e47578bc473a4f5e558bb/users/0de35369e91d4f58a8066cfc7f5d96a2/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:03.965888 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d2a2a-ebb4-4444-9a10-330f9bc1e56d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ed9f8de7f9e47578bc473a4f5e558bb, user_id=0de35369e91d4f58a8066cfc7f5d96a2, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:03.968534 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d2a2a-ebb4-4444-9a10-330f9bc1e56d admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:04.066057 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1267/2549] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:14:03 2022] PUT /identity/v3/projects/9ed9f8de7f9e47578bc473a4f5e558bb/users/0de35369e91d4f58a8066cfc7f5d96a2/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:14:04.077813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d2ca6-2fd9-493e-8f27-6b08a1027435 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:04.078104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d2ca6-2fd9-493e-8f27-6b08a1027435 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:04.078374 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d2ca6-2fd9-493e-8f27-6b08a1027435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:04.107019 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-bf7d2ca6-2fd9-493e-8f27-6b08a1027435 None None] MFA Rules not processed for user `0de35369e91d4f58a8066cfc7f5d96a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:04.108333 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bf7d2ca6-2fd9-493e-8f27-6b08a1027435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:04.170530 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1283/2550] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:14:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:14:04.175008 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d2a2a-ebb4-4444-9a10-330f9bc1e56d admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:04.234517 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d2a2a-ebb4-4444-9a10-330f9bc1e56d admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:04.239326 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0130a2-4154-4111-adf7-758394f5c616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:04.239944 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0130a2-4154-4111-adf7-758394f5c616 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:04.240258 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0130a2-4154-4111-adf7-758394f5c616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:04.240573 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0130a2-4154-4111-adf7-758394f5c616 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:04.241161 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0130a2-4154-4111-adf7-758394f5c616 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:04.243112 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0130a2-4154-4111-adf7-758394f5c616 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:04.318630 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1268/2551] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:14:04 2022] POST /identity/v3/projects => generated 363 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:14:04.322777 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d2ca6-2fd9-493e-8f27-6b08a1027435 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:04.333769 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d2ca6-2fd9-493e-8f27-6b08a1027435 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:04.393708 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155cab7-9c94-480b-b966-3be4a53035c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:04.394317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155cab7-9c94-480b-b966-3be4a53035c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:04.394602 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155cab7-9c94-480b-b966-3be4a53035c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:04.394878 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155cab7-9c94-480b-b966-3be4a53035c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:04.395515 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f155cab7-9c94-480b-b966-3be4a53035c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:04.397541 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f155cab7-9c94-480b-b966-3be4a53035c1 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:04.482399 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1284/2552] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:14:04 2022] POST /identity/v3/users => generated 288 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:14:04.492206 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0130a2-4154-4111-adf7-758394f5c616 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:04.502582 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0130a2-4154-4111-adf7-758394f5c616 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:04.508269 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8d25c0-025d-47e1-b120-661c3354a3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:04.508871 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d25c0-025d-47e1-b120-661c3354a3b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:04.509281 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d25c0-025d-47e1-b120-661c3354a3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:04.509557 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d25c0-025d-47e1-b120-661c3354a3b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:04.510258 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8d25c0-025d-47e1-b120-661c3354a3b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:04.511808 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8d25c0-025d-47e1-b120-661c3354a3b3 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:04.522262 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1269/2553] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:14:04.526606 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155cab7-9c94-480b-b966-3be4a53035c1 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:04.536604 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155cab7-9c94-480b-b966-3be4a53035c1 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:04.541192 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63261c7-d39a-4080-a4b7-f32d18bfbe66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:04.541834 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63261c7-d39a-4080-a4b7-f32d18bfbe66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:04.542116 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63261c7-d39a-4080-a4b7-f32d18bfbe66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:04.542392 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63261c7-d39a-4080-a4b7-f32d18bfbe66 admin admin] PATH_INFO: `/v3/projects/a129b2b059b0436cbb59d5f396f67de9/users/065a8cd41fd648229d5e08e6761dc681/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:04.557229 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63261c7-d39a-4080-a4b7-f32d18bfbe66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a129b2b059b0436cbb59d5f396f67de9, user_id=065a8cd41fd648229d5e08e6761dc681, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:04.559874 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63261c7-d39a-4080-a4b7-f32d18bfbe66 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:04.609208 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1285/2554] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:14:04 2022] PUT /identity/v3/projects/a129b2b059b0436cbb59d5f396f67de9/users/065a8cd41fd648229d5e08e6761dc681/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:14:04.613275 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8d25c0-025d-47e1-b120-661c3354a3b3 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:04.623797 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8d25c0-025d-47e1-b120-661c3354a3b3 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:04.647177 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:04.683907 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d70fd79-e640-4189-8f25-dc68f14a88a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:04.684542 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70fd79-e640-4189-8f25-dc68f14a88a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:04.684823 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70fd79-e640-4189-8f25-dc68f14a88a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:04.685118 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70fd79-e640-4189-8f25-dc68f14a88a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:04.685756 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d70fd79-e640-4189-8f25-dc68f14a88a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:04.687322 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d70fd79-e640-4189-8f25-dc68f14a88a1 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:04.696882 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1270/2555] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:14:04.701167 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63261c7-d39a-4080-a4b7-f32d18bfbe66 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:04.707003 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:04.712478 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63261c7-d39a-4080-a4b7-f32d18bfbe66 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:04.717589 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ed07c-bf99-430c-b705-573666e4f4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:04.718259 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ed07c-bf99-430c-b705-573666e4f4e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:04.718525 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ed07c-bf99-430c-b705-573666e4f4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:04.718804 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ed07c-bf99-430c-b705-573666e4f4e8 admin admin] PATH_INFO: `/v3/projects/a129b2b059b0436cbb59d5f396f67de9/users/065a8cd41fd648229d5e08e6761dc681/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:04.720754 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ed07c-bf99-430c-b705-573666e4f4e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a129b2b059b0436cbb59d5f396f67de9, user_id=065a8cd41fd648229d5e08e6761dc681, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:04.723478 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ed07c-bf99-430c-b705-573666e4f4e8 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:04.734342 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1286/2556] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:14:04 2022] PUT /identity/v3/projects/a129b2b059b0436cbb59d5f396f67de9/users/065a8cd41fd648229d5e08e6761dc681/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:14:04.745160 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523634f-93b2-49d4-8916-fc8fb5bde1dd None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:04.748728 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523634f-93b2-49d4-8916-fc8fb5bde1dd None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:04.748728 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523634f-93b2-49d4-8916-fc8fb5bde1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:04.773016 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-4523634f-93b2-49d4-8916-fc8fb5bde1dd None None] MFA Rules not processed for user `065a8cd41fd648229d5e08e6761dc681`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:04.777423 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-4523634f-93b2-49d4-8916-fc8fb5bde1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:04.840738 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1271/2557] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:14:04 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:14:04.853264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ed07c-bf99-430c-b705-573666e4f4e8 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:04.854935 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c46ed07c-bf99-430c-b705-573666e4f4e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:04.912582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ed07c-bf99-430c-b705-573666e4f4e8 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:04.916283 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34136e39-ebbf-4b25-9900-a21f9556da54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:04.916884 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34136e39-ebbf-4b25-9900-a21f9556da54 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:04.917166 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34136e39-ebbf-4b25-9900-a21f9556da54 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:04.917439 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34136e39-ebbf-4b25-9900-a21f9556da54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:04.926448 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34136e39-ebbf-4b25-9900-a21f9556da54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:04.928563 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34136e39-ebbf-4b25-9900-a21f9556da54 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:04.937815 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1287/2558] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:14:04 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:14:05.115782 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523634f-93b2-49d4-8916-fc8fb5bde1dd None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:05.125771 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523634f-93b2-49d4-8916-fc8fb5bde1dd None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:05.129231 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f579d-34cd-4e64-b647-56e38d6317fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:05.129822 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f579d-34cd-4e64-b647-56e38d6317fa service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:05.130154 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f579d-34cd-4e64-b647-56e38d6317fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:05.130446 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f579d-34cd-4e64-b647-56e38d6317fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:05.185956 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8f579d-34cd-4e64-b647-56e38d6317fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:05.187394 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8f579d-34cd-4e64-b647-56e38d6317fa service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:05.196238 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1272/2559] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:14:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:14:08.589993 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34136e39-ebbf-4b25-9900-a21f9556da54 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:08.653067 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34136e39-ebbf-4b25-9900-a21f9556da54 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:08.658177 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f632a-4732-433f-b968-43c38b2879ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:08.658812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f632a-4732-433f-b968-43c38b2879ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:08.659094 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f632a-4732-433f-b968-43c38b2879ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:08.659366 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f632a-4732-433f-b968-43c38b2879ae admin admin] PATH_INFO: `/v3/users/0de35369e91d4f58a8066cfc7f5d96a2` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:08.660532 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f632a-4732-433f-b968-43c38b2879ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=0de35369e91d4f58a8066cfc7f5d96a2)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:08.662494 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f632a-4732-433f-b968-43c38b2879ae admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:08.856703 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-985f632a-4732-433f-b968-43c38b2879ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0de35369e91d4f58a8066cfc7f5d96a2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:08.857263 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-985f632a-4732-433f-b968-43c38b2879ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0de35369e91d4f58a8066cfc7f5d96a2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:08.857857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-985f632a-4732-433f-b968-43c38b2879ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0de35369e91d4f58a8066cfc7f5d96a2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:08.865251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-985f632a-4732-433f-b968-43c38b2879ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0de35369e91d4f58a8066cfc7f5d96a2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:08.877313 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-985f632a-4732-433f-b968-43c38b2879ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0de35369e91d4f58a8066cfc7f5d96a2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:08.946549 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1288/2560] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:14:08 2022] DELETE /identity/v3/users/0de35369e91d4f58a8066cfc7f5d96a2 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:14:08.950752 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b718f2-806b-4469-ba9a-17a8f866aaf5 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:08.951221 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b718f2-806b-4469-ba9a-17a8f866aaf5 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:08.951318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b718f2-806b-4469-ba9a-17a8f866aaf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:08.970028 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-d0b718f2-806b-4469-ba9a-17a8f866aaf5 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:08.971226 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d0b718f2-806b-4469-ba9a-17a8f866aaf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:09.012532 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:09.030301 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1273/2561] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:14:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:14:09.039922 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f632a-4732-433f-b968-43c38b2879ae admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:09.041407 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-985f632a-4732-433f-b968-43c38b2879ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:09.055253 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:09.099971 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f632a-4732-433f-b968-43c38b2879ae admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:09.103613 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a63c692-a2aa-4f28-bfb6-e6bbf330b4f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:09.104251 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63c692-a2aa-4f28-bfb6-e6bbf330b4f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:09.104528 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63c692-a2aa-4f28-bfb6-e6bbf330b4f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:09.104864 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63c692-a2aa-4f28-bfb6-e6bbf330b4f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:09.126794 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a63c692-a2aa-4f28-bfb6-e6bbf330b4f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:09.126794 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a63c692-a2aa-4f28-bfb6-e6bbf330b4f8 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:09.134192 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1289/2562] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:14:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:14:09.535650 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b718f2-806b-4469-ba9a-17a8f866aaf5 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:09.537151 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d0b718f2-806b-4469-ba9a-17a8f866aaf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:09.552423 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b718f2-806b-4469-ba9a-17a8f866aaf5 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:09.557206 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99bfe4b-8119-436b-975e-6333d7f19c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:09.557852 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99bfe4b-8119-436b-975e-6333d7f19c77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:09.558092 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99bfe4b-8119-436b-975e-6333d7f19c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:09.558417 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99bfe4b-8119-436b-975e-6333d7f19c77 admin admin] PATH_INFO: `/v3/projects/9ed9f8de7f9e47578bc473a4f5e558bb` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:09.559489 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99bfe4b-8119-436b-975e-6333d7f19c77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ed9f8de7f9e47578bc473a4f5e558bb)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:09.561397 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99bfe4b-8119-436b-975e-6333d7f19c77 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:09.695473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a99bfe4b-8119-436b-975e-6333d7f19c77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ed9f8de7f9e47578bc473a4f5e558bb', 'request_id': 'req-a99bfe4b-8119-436b-975e-6333d7f19c77'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:09.768842 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a99bfe4b-8119-436b-975e-6333d7f19c77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ed9f8de7f9e47578bc473a4f5e558bb', 'request_id': 'req-a99bfe4b-8119-436b-975e-6333d7f19c77'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:09.775718 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a99bfe4b-8119-436b-975e-6333d7f19c77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ed9f8de7f9e47578bc473a4f5e558bb', 'request_id': 'req-a99bfe4b-8119-436b-975e-6333d7f19c77'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:09.785960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a99bfe4b-8119-436b-975e-6333d7f19c77 admin admin] The token cache is being invalidate because project 9ed9f8de7f9e47578bc473a4f5e558bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:14:09.786272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a99bfe4b-8119-436b-975e-6333d7f19c77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:09.786900 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a99bfe4b-8119-436b-975e-6333d7f19c77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:09.788388 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1274/2563] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:14:09 2022] DELETE /identity/v3/projects/9ed9f8de7f9e47578bc473a4f5e558bb => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:14:09.792474 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a63c692-a2aa-4f28-bfb6-e6bbf330b4f8 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:09.802126 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a63c692-a2aa-4f28-bfb6-e6bbf330b4f8 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:09.982246 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7225c5c-3656-4a6b-82f5-342a501f266e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:09.982770 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7225c5c-3656-4a6b-82f5-342a501f266e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:09.983049 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7225c5c-3656-4a6b-82f5-342a501f266e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:09.983321 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7225c5c-3656-4a6b-82f5-342a501f266e admin admin] PATH_INFO: `/v3/users/065a8cd41fd648229d5e08e6761dc681` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:09.984536 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7225c5c-3656-4a6b-82f5-342a501f266e admin admin] RBAC: Authorizing `identity:delete_user(user_id=065a8cd41fd648229d5e08e6761dc681)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:09.986443 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7225c5c-3656-4a6b-82f5-342a501f266e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:10.130379 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f7225c5c-3656-4a6b-82f5-342a501f266e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '065a8cd41fd648229d5e08e6761dc681'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:10.131031 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f7225c5c-3656-4a6b-82f5-342a501f266e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '065a8cd41fd648229d5e08e6761dc681'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:10.131636 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f7225c5c-3656-4a6b-82f5-342a501f266e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '065a8cd41fd648229d5e08e6761dc681'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:10.138520 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f7225c5c-3656-4a6b-82f5-342a501f266e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '065a8cd41fd648229d5e08e6761dc681'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:10.150513 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f7225c5c-3656-4a6b-82f5-342a501f266e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '065a8cd41fd648229d5e08e6761dc681'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:10.195558 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1290/2564] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:14:09 2022] DELETE /identity/v3/users/065a8cd41fd648229d5e08e6761dc681 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:14:10.570260 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99bfe4b-8119-436b-975e-6333d7f19c77 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:10.571802 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a99bfe4b-8119-436b-975e-6333d7f19c77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:10.586963 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99bfe4b-8119-436b-975e-6333d7f19c77 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:10.644767 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3442dc93-0f5e-4f73-89b3-011562f680c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:10.645396 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442dc93-0f5e-4f73-89b3-011562f680c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:10.645673 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442dc93-0f5e-4f73-89b3-011562f680c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:10.645943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442dc93-0f5e-4f73-89b3-011562f680c2 admin admin] PATH_INFO: `/v3/projects/a129b2b059b0436cbb59d5f396f67de9` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:10.647652 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3442dc93-0f5e-4f73-89b3-011562f680c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a129b2b059b0436cbb59d5f396f67de9)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:10.649588 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3442dc93-0f5e-4f73-89b3-011562f680c2 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:10.793931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3442dc93-0f5e-4f73-89b3-011562f680c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a129b2b059b0436cbb59d5f396f67de9', 'request_id': 'req-3442dc93-0f5e-4f73-89b3-011562f680c2'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:10.843005 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3442dc93-0f5e-4f73-89b3-011562f680c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a129b2b059b0436cbb59d5f396f67de9', 'request_id': 'req-3442dc93-0f5e-4f73-89b3-011562f680c2'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:10.849727 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3442dc93-0f5e-4f73-89b3-011562f680c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a129b2b059b0436cbb59d5f396f67de9', 'request_id': 'req-3442dc93-0f5e-4f73-89b3-011562f680c2'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:10.856167 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:10.860753 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3442dc93-0f5e-4f73-89b3-011562f680c2 admin admin] The token cache is being invalidate because project a129b2b059b0436cbb59d5f396f67de9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:14:10.861132 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3442dc93-0f5e-4f73-89b3-011562f680c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:10.861804 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3442dc93-0f5e-4f73-89b3-011562f680c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:10.863309 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1275/2565] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:14:10 2022] DELETE /identity/v3/projects/a129b2b059b0436cbb59d5f396f67de9 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:14:10.868124 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe30f96-a364-4a93-9188-be5abdba3c84 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:10.868638 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe30f96-a364-4a93-9188-be5abdba3c84 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:10.868748 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe30f96-a364-4a93-9188-be5abdba3c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:10.893980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-0fe30f96-a364-4a93-9188-be5abdba3c84 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:10.895623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0fe30f96-a364-4a93-9188-be5abdba3c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:10.933853 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:10.965755 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1291/2566] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:14:10.973558 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8549e4-a042-4cdc-98f8-9890818bc2a8 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:10.973869 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8549e4-a042-4cdc-98f8-9890818bc2a8 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:10.974140 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8549e4-a042-4cdc-98f8-9890818bc2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:10.994132 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-ae8549e4-a042-4cdc-98f8-9890818bc2a8 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:10.995355 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ae8549e4-a042-4cdc-98f8-9890818bc2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:11.003780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1276/2567] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:14:11.008822 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe30f96-a364-4a93-9188-be5abdba3c84 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:11.020915 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe30f96-a364-4a93-9188-be5abdba3c84 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:11.025874 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ad703-38fb-4f22-aff6-aa68c4746832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:11.026475 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ad703-38fb-4f22-aff6-aa68c4746832 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:11.026750 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ad703-38fb-4f22-aff6-aa68c4746832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:11.027018 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ad703-38fb-4f22-aff6-aa68c4746832 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:11.027662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ad703-38fb-4f22-aff6-aa68c4746832 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:14:11.027965 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ad703-38fb-4f22-aff6-aa68c4746832 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:11.029872 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ad703-38fb-4f22-aff6-aa68c4746832 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:11.045876 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1292/2568] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:14:11.050441 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8549e4-a042-4cdc-98f8-9890818bc2a8 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:11.060262 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8549e4-a042-4cdc-98f8-9890818bc2a8 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:11.064983 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a523839-3bcc-4d8a-97c1-ee1338c1eb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:11.065570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a523839-3bcc-4d8a-97c1-ee1338c1eb84 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:11.065855 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a523839-3bcc-4d8a-97c1-ee1338c1eb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:11.066128 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a523839-3bcc-4d8a-97c1-ee1338c1eb84 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:11.066756 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a523839-3bcc-4d8a-97c1-ee1338c1eb84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:11.069312 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a523839-3bcc-4d8a-97c1-ee1338c1eb84 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:11.183025 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1277/2569] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:14:11 2022] POST /identity/v3/projects => generated 379 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:14:11.187203 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ad703-38fb-4f22-aff6-aa68c4746832 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:11.197830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ad703-38fb-4f22-aff6-aa68c4746832 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:11.259034 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c3a00-1303-454b-abb7-fdd048a11d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:11.259866 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c3a00-1303-454b-abb7-fdd048a11d47 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:11.260289 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c3a00-1303-454b-abb7-fdd048a11d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:11.260669 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c3a00-1303-454b-abb7-fdd048a11d47 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:11.261562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83c3a00-1303-454b-abb7-fdd048a11d47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:11.264392 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83c3a00-1303-454b-abb7-fdd048a11d47 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:11.386838 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1293/2570] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:14:11 2022] POST /identity/v3/users => generated 290 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:14:11.391181 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a523839-3bcc-4d8a-97c1-ee1338c1eb84 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:11.401394 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a523839-3bcc-4d8a-97c1-ee1338c1eb84 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:11.405896 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a179fb-c1af-4cb8-843e-2c8f690c8074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:11.406485 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a179fb-c1af-4cb8-843e-2c8f690c8074 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:11.406761 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a179fb-c1af-4cb8-843e-2c8f690c8074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:11.407061 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a179fb-c1af-4cb8-843e-2c8f690c8074 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:11.407703 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a179fb-c1af-4cb8-843e-2c8f690c8074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:11.409312 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a179fb-c1af-4cb8-843e-2c8f690c8074 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:11.419402 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1278/2571] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:14:11.425923 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c3a00-1303-454b-abb7-fdd048a11d47 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:11.436770 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c3a00-1303-454b-abb7-fdd048a11d47 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:11.441933 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e588f0-0e7e-4619-8738-1b69f9eaeb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:11.442558 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e588f0-0e7e-4619-8738-1b69f9eaeb2a admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:11.442837 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e588f0-0e7e-4619-8738-1b69f9eaeb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:11.443109 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e588f0-0e7e-4619-8738-1b69f9eaeb2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:11.443752 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e588f0-0e7e-4619-8738-1b69f9eaeb2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:11.445398 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e588f0-0e7e-4619-8738-1b69f9eaeb2a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:11.455551 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1294/2572] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:14:11.459653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a179fb-c1af-4cb8-843e-2c8f690c8074 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:11.469813 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a179fb-c1af-4cb8-843e-2c8f690c8074 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:11.474938 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7b944-44ad-4537-9155-f9b810c1456b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:11.475577 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7b944-44ad-4537-9155-f9b810c1456b admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:11.475855 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7b944-44ad-4537-9155-f9b810c1456b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:11.476150 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7b944-44ad-4537-9155-f9b810c1456b admin admin] PATH_INFO: `/v3/projects/858954ca8d304bb689d914769a5372f6/users/4f517bc77f4a49f8a2e27678db671515/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:11.490878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab7b944-44ad-4537-9155-f9b810c1456b admin admin] RBAC: Authorizing `identity:create_grant(project_id=858954ca8d304bb689d914769a5372f6, user_id=4f517bc77f4a49f8a2e27678db671515, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:11.493553 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab7b944-44ad-4537-9155-f9b810c1456b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:11.551374 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1279/2573] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:14:11 2022] PUT /identity/v3/projects/858954ca8d304bb689d914769a5372f6/users/4f517bc77f4a49f8a2e27678db671515/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:14:11.564859 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e793c-7a2e-499e-8082-b06b101c6ed7 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:11.565151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e793c-7a2e-499e-8082-b06b101c6ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:11.565431 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e793c-7a2e-499e-8082-b06b101c6ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:11.592695 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-456e793c-7a2e-499e-8082-b06b101c6ed7 None None] MFA Rules not processed for user `4f517bc77f4a49f8a2e27678db671515`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:11.593907 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-456e793c-7a2e-499e-8082-b06b101c6ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:11.657014 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1295/2574] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:14:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:14:11.661579 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7b944-44ad-4537-9155-f9b810c1456b admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:11.725124 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7b944-44ad-4537-9155-f9b810c1456b admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:11.729830 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc5bbde-4486-4a26-b862-85d64244a0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:11.730419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc5bbde-4486-4a26-b862-85d64244a0fb admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:11.731417 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc5bbde-4486-4a26-b862-85d64244a0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:11.731417 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc5bbde-4486-4a26-b862-85d64244a0fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:11.731703 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc5bbde-4486-4a26-b862-85d64244a0fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:11.733614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc5bbde-4486-4a26-b862-85d64244a0fb admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:11.816511 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1280/2575] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:14:11 2022] POST /identity/v3/projects => generated 379 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:14:11.820685 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e793c-7a2e-499e-8082-b06b101c6ed7 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:11.831725 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e793c-7a2e-499e-8082-b06b101c6ed7 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:11.861058 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:11.894248 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30afdef2-c7d8-4792-ad78-edd58256cd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:11.895073 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afdef2-c7d8-4792-ad78-edd58256cd81 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:11.895457 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afdef2-c7d8-4792-ad78-edd58256cd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:11.895870 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afdef2-c7d8-4792-ad78-edd58256cd81 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:11.897299 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30afdef2-c7d8-4792-ad78-edd58256cd81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:11.900166 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30afdef2-c7d8-4792-ad78-edd58256cd81 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:11.963772 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1296/2576] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:14:11 2022] POST /identity/v3/users => generated 290 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:14:11.967796 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc5bbde-4486-4a26-b862-85d64244a0fb admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:11.978811 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc5bbde-4486-4a26-b862-85d64244a0fb admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:11.983445 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf40a81-c836-4c0a-a971-c5959d09f3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:11.984040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf40a81-c836-4c0a-a971-c5959d09f3b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:11.984340 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf40a81-c836-4c0a-a971-c5959d09f3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:11.984605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf40a81-c836-4c0a-a971-c5959d09f3b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:11.985246 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf40a81-c836-4c0a-a971-c5959d09f3b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:11.986809 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf40a81-c836-4c0a-a971-c5959d09f3b4 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:11.997394 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1281/2577] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:14:12.001590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30afdef2-c7d8-4792-ad78-edd58256cd81 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:12.011721 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30afdef2-c7d8-4792-ad78-edd58256cd81 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:12.016485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12877a-1810-4bed-966e-3cb52b8a2f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:12.017112 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12877a-1810-4bed-966e-3cb52b8a2f83 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:12.017387 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12877a-1810-4bed-966e-3cb52b8a2f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:12.017658 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12877a-1810-4bed-966e-3cb52b8a2f83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:12.018314 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d12877a-1810-4bed-966e-3cb52b8a2f83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:12.019908 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d12877a-1810-4bed-966e-3cb52b8a2f83 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:12.030730 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1297/2578] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:14:12.034978 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf40a81-c836-4c0a-a971-c5959d09f3b4 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:12.045761 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf40a81-c836-4c0a-a971-c5959d09f3b4 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:12.050642 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed868cc8-0ad3-49e3-a8ba-ed00a28a195f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:12.051283 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed868cc8-0ad3-49e3-a8ba-ed00a28a195f admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:12.051565 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed868cc8-0ad3-49e3-a8ba-ed00a28a195f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:12.051837 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed868cc8-0ad3-49e3-a8ba-ed00a28a195f admin admin] PATH_INFO: `/v3/projects/f258713067e4483fbe419a22a4616d07/users/e2d111bec82a4848bcc49be5daa1fbf6/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:12.066063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed868cc8-0ad3-49e3-a8ba-ed00a28a195f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f258713067e4483fbe419a22a4616d07, user_id=e2d111bec82a4848bcc49be5daa1fbf6, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:12.068747 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed868cc8-0ad3-49e3-a8ba-ed00a28a195f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:12.125085 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1282/2579] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:14:12 2022] PUT /identity/v3/projects/f258713067e4483fbe419a22a4616d07/users/e2d111bec82a4848bcc49be5daa1fbf6/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:14:12.135807 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b2ef3-0423-4e31-ae69-880975a7fe76 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:12.136126 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b2ef3-0423-4e31-ae69-880975a7fe76 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:12.136438 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b2ef3-0423-4e31-ae69-880975a7fe76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:12.169434 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-359b2ef3-0423-4e31-ae69-880975a7fe76 None None] MFA Rules not processed for user `e2d111bec82a4848bcc49be5daa1fbf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:12.170645 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-359b2ef3-0423-4e31-ae69-880975a7fe76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:12.249626 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1298/2580] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:14:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:14:12.254100 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed868cc8-0ad3-49e3-a8ba-ed00a28a195f admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:12.298987 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:12.315298 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed868cc8-0ad3-49e3-a8ba-ed00a28a195f admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:12.320565 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc9b3e-1ba5-40e0-bec9-62ec712ee36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:12.321165 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc9b3e-1ba5-40e0-bec9-62ec712ee36b admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:12.321443 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc9b3e-1ba5-40e0-bec9-62ec712ee36b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:12.321714 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc9b3e-1ba5-40e0-bec9-62ec712ee36b admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:12.322340 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc9b3e-1ba5-40e0-bec9-62ec712ee36b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:12.324261 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc9b3e-1ba5-40e0-bec9-62ec712ee36b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:12.421568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1283/2581] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:14:12 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:14:12.425552 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b2ef3-0423-4e31-ae69-880975a7fe76 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:12.436083 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b2ef3-0423-4e31-ae69-880975a7fe76 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:12.495129 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd35653-f242-4018-885c-b30d19072e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:12.495740 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd35653-f242-4018-885c-b30d19072e41 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:12.496018 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd35653-f242-4018-885c-b30d19072e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:12.496323 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd35653-f242-4018-885c-b30d19072e41 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:12.496964 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd35653-f242-4018-885c-b30d19072e41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:12.498884 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd35653-f242-4018-885c-b30d19072e41 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:12.586344 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1299/2582] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:14:12 2022] POST /identity/v3/users => generated 296 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:14:12.590310 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc9b3e-1ba5-40e0-bec9-62ec712ee36b admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:12.601022 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc9b3e-1ba5-40e0-bec9-62ec712ee36b admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:12.606703 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df2029-a2e6-4b1f-9e11-3358e10dd219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:12.608682 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df2029-a2e6-4b1f-9e11-3358e10dd219 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:12.608682 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df2029-a2e6-4b1f-9e11-3358e10dd219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:12.608682 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df2029-a2e6-4b1f-9e11-3358e10dd219 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:12.609182 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df2029-a2e6-4b1f-9e11-3358e10dd219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:12.610832 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df2029-a2e6-4b1f-9e11-3358e10dd219 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:12.622753 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1284/2583] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:14:12.627105 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd35653-f242-4018-885c-b30d19072e41 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:12.637776 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd35653-f242-4018-885c-b30d19072e41 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:12.642509 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21d744-a245-4cd1-851d-725448e4cb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:12.643164 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21d744-a245-4cd1-851d-725448e4cb86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:12.643633 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21d744-a245-4cd1-851d-725448e4cb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:12.643913 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21d744-a245-4cd1-851d-725448e4cb86 admin admin] PATH_INFO: `/v3/projects/fe5f01e9e25d4fd2892b2ceb8d44b44e/users/9ae73f13ed214fee9235721fa00f40a8/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:12.662507 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e21d744-a245-4cd1-851d-725448e4cb86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe5f01e9e25d4fd2892b2ceb8d44b44e, user_id=9ae73f13ed214fee9235721fa00f40a8, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:12.665202 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e21d744-a245-4cd1-851d-725448e4cb86 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:12.723377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1300/2584] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:14:12 2022] PUT /identity/v3/projects/fe5f01e9e25d4fd2892b2ceb8d44b44e/users/9ae73f13ed214fee9235721fa00f40a8/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:14:12.727661 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df2029-a2e6-4b1f-9e11-3358e10dd219 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:12.737886 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df2029-a2e6-4b1f-9e11-3358e10dd219 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:12.796705 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d65df-a2a2-48f2-8950-c2d7dcedcd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:12.797323 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d65df-a2a2-48f2-8950-c2d7dcedcd8a admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:12.797654 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d65df-a2a2-48f2-8950-c2d7dcedcd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:12.797928 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d65df-a2a2-48f2-8950-c2d7dcedcd8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:12.798565 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d65df-a2a2-48f2-8950-c2d7dcedcd8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:12.800178 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d65df-a2a2-48f2-8950-c2d7dcedcd8a admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:12.810553 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1285/2585] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:14:12.814795 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21d744-a245-4cd1-851d-725448e4cb86 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:12.821031 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:12.826271 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21d744-a245-4cd1-851d-725448e4cb86 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:12.831185 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46a6bd-e323-4352-a9ff-0b746e3c2e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:12.831833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46a6bd-e323-4352-a9ff-0b746e3c2e2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:12.832145 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46a6bd-e323-4352-a9ff-0b746e3c2e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:12.832423 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46a6bd-e323-4352-a9ff-0b746e3c2e2f admin admin] PATH_INFO: `/v3/projects/fe5f01e9e25d4fd2892b2ceb8d44b44e/users/9ae73f13ed214fee9235721fa00f40a8/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:12.834364 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46a6bd-e323-4352-a9ff-0b746e3c2e2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe5f01e9e25d4fd2892b2ceb8d44b44e, user_id=9ae73f13ed214fee9235721fa00f40a8, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:12.837169 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46a6bd-e323-4352-a9ff-0b746e3c2e2f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:12.847566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1301/2586] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:14:12 2022] PUT /identity/v3/projects/fe5f01e9e25d4fd2892b2ceb8d44b44e/users/9ae73f13ed214fee9235721fa00f40a8/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:14:12.859065 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e48bcd-dae4-4372-80cd-ab3672c62b88 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:12.859542 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e48bcd-dae4-4372-80cd-ab3672c62b88 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:12.859816 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e48bcd-dae4-4372-80cd-ab3672c62b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:12.892765 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-48e48bcd-dae4-4372-80cd-ab3672c62b88 None None] MFA Rules not processed for user `9ae73f13ed214fee9235721fa00f40a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:12.893980 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-48e48bcd-dae4-4372-80cd-ab3672c62b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:12.967039 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1286/2587] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:14:12 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:14:12.981453 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46a6bd-e323-4352-a9ff-0b746e3c2e2f admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:12.983302 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-db46a6bd-e323-4352-a9ff-0b746e3c2e2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:13.046569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46a6bd-e323-4352-a9ff-0b746e3c2e2f admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:13.053342 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a87941-38ab-414f-adb2-f084665e315d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:13.053960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a87941-38ab-414f-adb2-f084665e315d service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:13.054354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a87941-38ab-414f-adb2-f084665e315d service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:13.057220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a87941-38ab-414f-adb2-f084665e315d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:13.074017 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a87941-38ab-414f-adb2-f084665e315d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:13.075460 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a87941-38ab-414f-adb2-f084665e315d service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:13.087793 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1302/2588] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:14:12 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:14:13.235554 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e48bcd-dae4-4372-80cd-ab3672c62b88 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:13.236998 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-48e48bcd-dae4-4372-80cd-ab3672c62b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:13.249488 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e48bcd-dae4-4372-80cd-ab3672c62b88 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:13.253030 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef5b0b-5214-4048-b9e4-37f5ea5a6fb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:13.253617 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef5b0b-5214-4048-b9e4-37f5ea5a6fb6 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:13.253896 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef5b0b-5214-4048-b9e4-37f5ea5a6fb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:13.254181 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef5b0b-5214-4048-b9e4-37f5ea5a6fb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:13.293197 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:13.311337 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef5b0b-5214-4048-b9e4-37f5ea5a6fb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:13.312812 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef5b0b-5214-4048-b9e4-37f5ea5a6fb6 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:13.322304 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1287/2589] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:14:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:14:17.086216 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a87941-38ab-414f-adb2-f084665e315d service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:17.148351 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a87941-38ab-414f-adb2-f084665e315d service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:17.153097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f92d0-fced-436c-aaf3-968153249eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:17.158299 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f92d0-fced-436c-aaf3-968153249eca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:17.159003 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f92d0-fced-436c-aaf3-968153249eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:17.159288 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f92d0-fced-436c-aaf3-968153249eca admin admin] PATH_INFO: `/v3/users/4f517bc77f4a49f8a2e27678db671515` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:17.160564 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002f92d0-fced-436c-aaf3-968153249eca admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f517bc77f4a49f8a2e27678db671515)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:17.163126 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002f92d0-fced-436c-aaf3-968153249eca admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:17.290485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-002f92d0-fced-436c-aaf3-968153249eca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f517bc77f4a49f8a2e27678db671515'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:17.291059 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-002f92d0-fced-436c-aaf3-968153249eca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f517bc77f4a49f8a2e27678db671515'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:17.291652 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-002f92d0-fced-436c-aaf3-968153249eca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f517bc77f4a49f8a2e27678db671515'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:17.298577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-002f92d0-fced-436c-aaf3-968153249eca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f517bc77f4a49f8a2e27678db671515'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:17.310347 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-002f92d0-fced-436c-aaf3-968153249eca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f517bc77f4a49f8a2e27678db671515'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:17.374256 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1303/2590] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:14:17 2022] DELETE /identity/v3/users/4f517bc77f4a49f8a2e27678db671515 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:14:17.378701 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f9be3-17b8-4efd-a4bd-f50d1b886c3c None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:17.379165 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f9be3-17b8-4efd-a4bd-f50d1b886c3c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:17.379282 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f9be3-17b8-4efd-a4bd-f50d1b886c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:17.398978 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-368f9be3-17b8-4efd-a4bd-f50d1b886c3c None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:17.400194 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-368f9be3-17b8-4efd-a4bd-f50d1b886c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:17.456347 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1288/2591] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:14:17.466643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f92d0-fced-436c-aaf3-968153249eca admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:17.468055 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-002f92d0-fced-436c-aaf3-968153249eca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:17.481506 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:17.524684 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f92d0-fced-436c-aaf3-968153249eca admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:17.528348 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b84cd6-9cef-47f7-9714-20a1b3d592e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:17.528943 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b84cd6-9cef-47f7-9714-20a1b3d592e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:17.529220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b84cd6-9cef-47f7-9714-20a1b3d592e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:17.529520 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b84cd6-9cef-47f7-9714-20a1b3d592e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:17.538724 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b84cd6-9cef-47f7-9714-20a1b3d592e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:17.540245 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b84cd6-9cef-47f7-9714-20a1b3d592e6 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:17.549670 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1304/2592] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:14:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:14:17.895301 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f9be3-17b8-4efd-a4bd-f50d1b886c3c None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:17.896860 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-368f9be3-17b8-4efd-a4bd-f50d1b886c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:17.910734 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f9be3-17b8-4efd-a4bd-f50d1b886c3c None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:17.915567 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acec9e6-2da8-44f9-9e38-242a6bce2512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:17.916941 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acec9e6-2da8-44f9-9e38-242a6bce2512 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:17.917402 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acec9e6-2da8-44f9-9e38-242a6bce2512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:17.917688 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acec9e6-2da8-44f9-9e38-242a6bce2512 admin admin] PATH_INFO: `/v3/projects/858954ca8d304bb689d914769a5372f6` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:17.918833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acec9e6-2da8-44f9-9e38-242a6bce2512 admin admin] RBAC: Authorizing `identity:delete_project(project_id=858954ca8d304bb689d914769a5372f6)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:17.925035 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acec9e6-2da8-44f9-9e38-242a6bce2512 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:18.015143 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0acec9e6-2da8-44f9-9e38-242a6bce2512 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '858954ca8d304bb689d914769a5372f6', 'request_id': 'req-0acec9e6-2da8-44f9-9e38-242a6bce2512'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:18.059825 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0acec9e6-2da8-44f9-9e38-242a6bce2512 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '858954ca8d304bb689d914769a5372f6', 'request_id': 'req-0acec9e6-2da8-44f9-9e38-242a6bce2512'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:18.066405 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0acec9e6-2da8-44f9-9e38-242a6bce2512 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '858954ca8d304bb689d914769a5372f6', 'request_id': 'req-0acec9e6-2da8-44f9-9e38-242a6bce2512'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:18.077454 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0acec9e6-2da8-44f9-9e38-242a6bce2512 admin admin] The token cache is being invalidate because project 858954ca8d304bb689d914769a5372f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:14:18.077745 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0acec9e6-2da8-44f9-9e38-242a6bce2512 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:18.078370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0acec9e6-2da8-44f9-9e38-242a6bce2512 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:18.080316 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1289/2593] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:14:17 2022] DELETE /identity/v3/projects/858954ca8d304bb689d914769a5372f6 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:14:18.084458 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b84cd6-9cef-47f7-9714-20a1b3d592e6 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:18.094002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b84cd6-9cef-47f7-9714-20a1b3d592e6 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:18.149451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba8054f-eb75-4891-94c1-c0a7d8e1ffb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:18.150111 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8054f-eb75-4891-94c1-c0a7d8e1ffb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:18.150392 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8054f-eb75-4891-94c1-c0a7d8e1ffb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:18.150667 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8054f-eb75-4891-94c1-c0a7d8e1ffb9 admin admin] PATH_INFO: `/v3/users/e2d111bec82a4848bcc49be5daa1fbf6` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:18.151782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba8054f-eb75-4891-94c1-c0a7d8e1ffb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2d111bec82a4848bcc49be5daa1fbf6)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:18.153771 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba8054f-eb75-4891-94c1-c0a7d8e1ffb9 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:18.330470 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9ba8054f-eb75-4891-94c1-c0a7d8e1ffb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2d111bec82a4848bcc49be5daa1fbf6'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:18.331048 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9ba8054f-eb75-4891-94c1-c0a7d8e1ffb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2d111bec82a4848bcc49be5daa1fbf6'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:18.331657 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9ba8054f-eb75-4891-94c1-c0a7d8e1ffb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2d111bec82a4848bcc49be5daa1fbf6'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:18.338557 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9ba8054f-eb75-4891-94c1-c0a7d8e1ffb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2d111bec82a4848bcc49be5daa1fbf6'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:18.350888 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9ba8054f-eb75-4891-94c1-c0a7d8e1ffb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2d111bec82a4848bcc49be5daa1fbf6'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:18.407935 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1305/2594] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:14:18 2022] DELETE /identity/v3/users/e2d111bec82a4848bcc49be5daa1fbf6 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:14:18.816514 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acec9e6-2da8-44f9-9e38-242a6bce2512 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:18.822003 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0acec9e6-2da8-44f9-9e38-242a6bce2512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:18.832499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acec9e6-2da8-44f9-9e38-242a6bce2512 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:18.889244 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd37b2-0130-4bfc-9834-e6c8c2b5b468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:18.889877 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd37b2-0130-4bfc-9834-e6c8c2b5b468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:18.890156 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd37b2-0130-4bfc-9834-e6c8c2b5b468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:18.890452 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd37b2-0130-4bfc-9834-e6c8c2b5b468 admin admin] PATH_INFO: `/v3/projects/f258713067e4483fbe419a22a4616d07` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:18.891639 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cd37b2-0130-4bfc-9834-e6c8c2b5b468 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f258713067e4483fbe419a22a4616d07)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:18.893754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cd37b2-0130-4bfc-9834-e6c8c2b5b468 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:18.896570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:18.996247 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-c2cd37b2-0130-4bfc-9834-e6c8c2b5b468 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f258713067e4483fbe419a22a4616d07', 'request_id': 'req-c2cd37b2-0130-4bfc-9834-e6c8c2b5b468'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:19.065597 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-c2cd37b2-0130-4bfc-9834-e6c8c2b5b468 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f258713067e4483fbe419a22a4616d07', 'request_id': 'req-c2cd37b2-0130-4bfc-9834-e6c8c2b5b468'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:19.072800 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-c2cd37b2-0130-4bfc-9834-e6c8c2b5b468 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f258713067e4483fbe419a22a4616d07', 'request_id': 'req-c2cd37b2-0130-4bfc-9834-e6c8c2b5b468'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:19.084945 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-c2cd37b2-0130-4bfc-9834-e6c8c2b5b468 admin admin] The token cache is being invalidate because project f258713067e4483fbe419a22a4616d07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:14:19.085233 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-c2cd37b2-0130-4bfc-9834-e6c8c2b5b468 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:19.085864 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-c2cd37b2-0130-4bfc-9834-e6c8c2b5b468 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:19.087423 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1290/2595] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:14:18 2022] DELETE /identity/v3/projects/f258713067e4483fbe419a22a4616d07 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:14:19.091400 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba8054f-eb75-4891-94c1-c0a7d8e1ffb9 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:19.092879 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9ba8054f-eb75-4891-94c1-c0a7d8e1ffb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:19.105425 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba8054f-eb75-4891-94c1-c0a7d8e1ffb9 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:19.135214 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:19.160770 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60390587-75f6-4b43-b63b-11a3a524b530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:19.161445 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60390587-75f6-4b43-b63b-11a3a524b530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:19.161724 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60390587-75f6-4b43-b63b-11a3a524b530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:19.162213 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60390587-75f6-4b43-b63b-11a3a524b530 admin admin] PATH_INFO: `/v3/users/9ae73f13ed214fee9235721fa00f40a8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:19.163717 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60390587-75f6-4b43-b63b-11a3a524b530 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ae73f13ed214fee9235721fa00f40a8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:19.165754 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60390587-75f6-4b43-b63b-11a3a524b530 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:19.364973 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-60390587-75f6-4b43-b63b-11a3a524b530 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ae73f13ed214fee9235721fa00f40a8'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:19.365595 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-60390587-75f6-4b43-b63b-11a3a524b530 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ae73f13ed214fee9235721fa00f40a8'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:19.366190 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-60390587-75f6-4b43-b63b-11a3a524b530 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ae73f13ed214fee9235721fa00f40a8'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:19.373206 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-60390587-75f6-4b43-b63b-11a3a524b530 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ae73f13ed214fee9235721fa00f40a8'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:19.385542 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-60390587-75f6-4b43-b63b-11a3a524b530 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ae73f13ed214fee9235721fa00f40a8'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:19.453433 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1306/2596] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:14:19 2022] DELETE /identity/v3/users/9ae73f13ed214fee9235721fa00f40a8 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:14:19.844133 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd37b2-0130-4bfc-9834-e6c8c2b5b468 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:19.845562 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c2cd37b2-0130-4bfc-9834-e6c8c2b5b468 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:19.860754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd37b2-0130-4bfc-9834-e6c8c2b5b468 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:19.917607 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa608b15-e2d8-4170-9fc8-dc8f3e39f6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:19.918164 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa608b15-e2d8-4170-9fc8-dc8f3e39f6b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:19.918445 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa608b15-e2d8-4170-9fc8-dc8f3e39f6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:19.918714 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa608b15-e2d8-4170-9fc8-dc8f3e39f6b7 admin admin] PATH_INFO: `/v3/projects/fe5f01e9e25d4fd2892b2ceb8d44b44e` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:19.919830 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa608b15-e2d8-4170-9fc8-dc8f3e39f6b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe5f01e9e25d4fd2892b2ceb8d44b44e)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:19.921743 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa608b15-e2d8-4170-9fc8-dc8f3e39f6b7 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:20.003621 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-aa608b15-e2d8-4170-9fc8-dc8f3e39f6b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe5f01e9e25d4fd2892b2ceb8d44b44e', 'request_id': 'req-aa608b15-e2d8-4170-9fc8-dc8f3e39f6b7'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:20.071121 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-aa608b15-e2d8-4170-9fc8-dc8f3e39f6b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe5f01e9e25d4fd2892b2ceb8d44b44e', 'request_id': 'req-aa608b15-e2d8-4170-9fc8-dc8f3e39f6b7'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:20.078405 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-aa608b15-e2d8-4170-9fc8-dc8f3e39f6b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe5f01e9e25d4fd2892b2ceb8d44b44e', 'request_id': 'req-aa608b15-e2d8-4170-9fc8-dc8f3e39f6b7'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:20.090022 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-aa608b15-e2d8-4170-9fc8-dc8f3e39f6b7 admin admin] The token cache is being invalidate because project fe5f01e9e25d4fd2892b2ceb8d44b44e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:14:20.090310 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-aa608b15-e2d8-4170-9fc8-dc8f3e39f6b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:20.090947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-aa608b15-e2d8-4170-9fc8-dc8f3e39f6b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:20.092449 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1291/2597] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:14:19 2022] DELETE /identity/v3/projects/fe5f01e9e25d4fd2892b2ceb8d44b44e => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:14:20.098443 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff360ed-4b05-48c5-bb1f-4d801f558477 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:20.098737 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff360ed-4b05-48c5-bb1f-4d801f558477 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:20.099661 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff360ed-4b05-48c5-bb1f-4d801f558477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:20.118853 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-bff360ed-4b05-48c5-bb1f-4d801f558477 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:20.120057 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bff360ed-4b05-48c5-bb1f-4d801f558477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:20.178692 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1307/2598] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:14:20.187035 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45849c87-4b80-421c-878f-4f9c80577171 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:20.187316 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45849c87-4b80-421c-878f-4f9c80577171 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:20.187588 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45849c87-4b80-421c-878f-4f9c80577171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:20.207567 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-45849c87-4b80-421c-878f-4f9c80577171 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:20.209292 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-45849c87-4b80-421c-878f-4f9c80577171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:20.216913 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1292/2599] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:14:20.220884 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff360ed-4b05-48c5-bb1f-4d801f558477 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:20.231064 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff360ed-4b05-48c5-bb1f-4d801f558477 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:20.235649 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f57ceb-0841-4e9a-97bf-277b85d37138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:20.236304 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f57ceb-0841-4e9a-97bf-277b85d37138 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:20.236590 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f57ceb-0841-4e9a-97bf-277b85d37138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:20.236860 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f57ceb-0841-4e9a-97bf-277b85d37138 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:20.237502 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f57ceb-0841-4e9a-97bf-277b85d37138 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:14:20.237807 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f57ceb-0841-4e9a-97bf-277b85d37138 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:20.239385 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f57ceb-0841-4e9a-97bf-277b85d37138 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:20.250866 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1308/2600] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:14:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:14:20.254867 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45849c87-4b80-421c-878f-4f9c80577171 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:20.263743 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45849c87-4b80-421c-878f-4f9c80577171 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:20.268397 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9a193-0639-490d-a5d5-cf12de58b2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:20.268970 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9a193-0639-490d-a5d5-cf12de58b2bd admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:20.269245 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9a193-0639-490d-a5d5-cf12de58b2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:20.269510 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9a193-0639-490d-a5d5-cf12de58b2bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:20.270122 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee9a193-0639-490d-a5d5-cf12de58b2bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:20.272584 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee9a193-0639-490d-a5d5-cf12de58b2bd admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:20.286343 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:20.358642 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1293/2601] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:14:20 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:14:20.362309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f57ceb-0841-4e9a-97bf-277b85d37138 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:20.372329 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f57ceb-0841-4e9a-97bf-277b85d37138 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:20.410514 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:20.428570 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea09fa4-0e20-4976-a1bc-87a549641cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:20.429378 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea09fa4-0e20-4976-a1bc-87a549641cb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:20.429522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea09fa4-0e20-4976-a1bc-87a549641cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:20.430349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea09fa4-0e20-4976-a1bc-87a549641cb2 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:20.430983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea09fa4-0e20-4976-a1bc-87a549641cb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:20.432953 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea09fa4-0e20-4976-a1bc-87a549641cb2 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:20.562445 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1309/2602] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:14:20 2022] POST /identity/v3/users => generated 292 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:14:20.566762 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9a193-0639-490d-a5d5-cf12de58b2bd admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:20.577493 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9a193-0639-490d-a5d5-cf12de58b2bd admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:20.582307 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd733c-7326-4dc1-9ec5-700f26928e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:20.582898 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd733c-7326-4dc1-9ec5-700f26928e94 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:20.583174 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd733c-7326-4dc1-9ec5-700f26928e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:20.584017 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd733c-7326-4dc1-9ec5-700f26928e94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:20.584692 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd733c-7326-4dc1-9ec5-700f26928e94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:20.586293 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd733c-7326-4dc1-9ec5-700f26928e94 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:20.596441 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1294/2603] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:14:20.600328 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea09fa4-0e20-4976-a1bc-87a549641cb2 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:20.610806 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea09fa4-0e20-4976-a1bc-87a549641cb2 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:20.615812 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29851608-dbc4-468b-b7be-fb4329013afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:20.616504 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29851608-dbc4-468b-b7be-fb4329013afc admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:20.616787 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29851608-dbc4-468b-b7be-fb4329013afc admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:20.617061 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29851608-dbc4-468b-b7be-fb4329013afc admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:20.617708 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29851608-dbc4-468b-b7be-fb4329013afc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:20.619358 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29851608-dbc4-468b-b7be-fb4329013afc admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:20.629953 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1310/2604] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:14:20.634475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd733c-7326-4dc1-9ec5-700f26928e94 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:20.644278 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd733c-7326-4dc1-9ec5-700f26928e94 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:20.649406 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e3d61c-952f-4d53-b9dc-f32e293162da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:20.650034 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3d61c-952f-4d53-b9dc-f32e293162da admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:20.650310 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3d61c-952f-4d53-b9dc-f32e293162da admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:20.650580 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3d61c-952f-4d53-b9dc-f32e293162da admin admin] PATH_INFO: `/v3/projects/9e17598bfa9c440d9fed85f3ec3cffc1/users/c3cec4d5a20247b290cc86de6bbbfbb5/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:20.664682 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e3d61c-952f-4d53-b9dc-f32e293162da admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e17598bfa9c440d9fed85f3ec3cffc1, user_id=c3cec4d5a20247b290cc86de6bbbfbb5, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:20.667296 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e3d61c-952f-4d53-b9dc-f32e293162da admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:20.733509 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1295/2605] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:14:20 2022] PUT /identity/v3/projects/9e17598bfa9c440d9fed85f3ec3cffc1/users/c3cec4d5a20247b290cc86de6bbbfbb5/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:14:20.745220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e19a0-28b3-4ea9-91db-fbcabeda46ff None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:20.745507 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e19a0-28b3-4ea9-91db-fbcabeda46ff None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:20.745776 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e19a0-28b3-4ea9-91db-fbcabeda46ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:20.775020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1b8e19a0-28b3-4ea9-91db-fbcabeda46ff None None] MFA Rules not processed for user `c3cec4d5a20247b290cc86de6bbbfbb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:20.776262 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1b8e19a0-28b3-4ea9-91db-fbcabeda46ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:20.840391 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1311/2606] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:14:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:14:20.844838 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e3d61c-952f-4d53-b9dc-f32e293162da admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:20.904867 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e3d61c-952f-4d53-b9dc-f32e293162da admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:20.909501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfc660-326b-4c7b-b83f-aa552abff0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:20.910090 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfc660-326b-4c7b-b83f-aa552abff0c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:20.910366 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfc660-326b-4c7b-b83f-aa552abff0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:20.910633 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfc660-326b-4c7b-b83f-aa552abff0c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:20.911266 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dfc660-326b-4c7b-b83f-aa552abff0c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:20.913193 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dfc660-326b-4c7b-b83f-aa552abff0c9 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:21.004178 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1296/2607] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:14:20 2022] POST /identity/v3/projects => generated 383 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:14:21.008928 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e19a0-28b3-4ea9-91db-fbcabeda46ff None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:21.019991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e19a0-28b3-4ea9-91db-fbcabeda46ff None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:21.075801 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c142b-dad2-454f-ae41-a03aa364fa52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:21.076448 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c142b-dad2-454f-ae41-a03aa364fa52 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:21.076730 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c142b-dad2-454f-ae41-a03aa364fa52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:21.077069 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c142b-dad2-454f-ae41-a03aa364fa52 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:21.077630 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c142b-dad2-454f-ae41-a03aa364fa52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:21.079813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c142b-dad2-454f-ae41-a03aa364fa52 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:21.191942 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1312/2608] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:14:21 2022] POST /identity/v3/users => generated 298 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:14:21.195995 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfc660-326b-4c7b-b83f-aa552abff0c9 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:21.206962 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfc660-326b-4c7b-b83f-aa552abff0c9 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:21.211835 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde09369-a104-42cf-9cad-ba0ddb23bd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:21.212480 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde09369-a104-42cf-9cad-ba0ddb23bd86 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:21.212763 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde09369-a104-42cf-9cad-ba0ddb23bd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:21.213034 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde09369-a104-42cf-9cad-ba0ddb23bd86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:21.213716 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde09369-a104-42cf-9cad-ba0ddb23bd86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:21.215301 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde09369-a104-42cf-9cad-ba0ddb23bd86 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:21.226685 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1297/2609] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:14:21.235025 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c142b-dad2-454f-ae41-a03aa364fa52 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:21.252061 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c142b-dad2-454f-ae41-a03aa364fa52 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:21.256794 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b67253-713d-42d1-b47b-62e9af097e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:21.257460 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b67253-713d-42d1-b47b-62e9af097e78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:21.257743 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b67253-713d-42d1-b47b-62e9af097e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:21.258016 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b67253-713d-42d1-b47b-62e9af097e78 admin admin] PATH_INFO: `/v3/projects/7e68d76f0c0e4b14b02a968aa80ec188/users/7cf8cc6b2166466dbc8a886fb9c4043e/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:21.273253 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b67253-713d-42d1-b47b-62e9af097e78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e68d76f0c0e4b14b02a968aa80ec188, user_id=7cf8cc6b2166466dbc8a886fb9c4043e, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:21.275912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b67253-713d-42d1-b47b-62e9af097e78 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:21.328700 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1313/2610] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:14:21 2022] PUT /identity/v3/projects/7e68d76f0c0e4b14b02a968aa80ec188/users/7cf8cc6b2166466dbc8a886fb9c4043e/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:14:21.333079 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde09369-a104-42cf-9cad-ba0ddb23bd86 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:21.342826 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde09369-a104-42cf-9cad-ba0ddb23bd86 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:21.404642 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3014d8ef-0470-4a31-8841-8b63bb26d94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:21.405247 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014d8ef-0470-4a31-8841-8b63bb26d94e admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:21.405521 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014d8ef-0470-4a31-8841-8b63bb26d94e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:21.405787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014d8ef-0470-4a31-8841-8b63bb26d94e admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:21.406416 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3014d8ef-0470-4a31-8841-8b63bb26d94e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:21.407985 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3014d8ef-0470-4a31-8841-8b63bb26d94e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:21.418628 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1298/2611] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:14:21.422855 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b67253-713d-42d1-b47b-62e9af097e78 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:21.434223 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b67253-713d-42d1-b47b-62e9af097e78 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:21.439083 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a7a8c-27a7-41e6-8b45-2747f6344d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:21.444566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a7a8c-27a7-41e6-8b45-2747f6344d8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:21.444566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a7a8c-27a7-41e6-8b45-2747f6344d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:21.444566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a7a8c-27a7-41e6-8b45-2747f6344d8e admin admin] PATH_INFO: `/v3/projects/7e68d76f0c0e4b14b02a968aa80ec188/users/7cf8cc6b2166466dbc8a886fb9c4043e/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:21.444566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a7a8c-27a7-41e6-8b45-2747f6344d8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e68d76f0c0e4b14b02a968aa80ec188, user_id=7cf8cc6b2166466dbc8a886fb9c4043e, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:21.445318 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a7a8c-27a7-41e6-8b45-2747f6344d8e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:21.451858 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:21.456767 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1314/2612] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:14:21 2022] PUT /identity/v3/projects/7e68d76f0c0e4b14b02a968aa80ec188/users/7cf8cc6b2166466dbc8a886fb9c4043e/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:14:21.467749 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1734e46-ffc2-419f-8221-4fe6134ddf73 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:21.468035 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1734e46-ffc2-419f-8221-4fe6134ddf73 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:21.468340 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1734e46-ffc2-419f-8221-4fe6134ddf73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:21.496451 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-a1734e46-ffc2-419f-8221-4fe6134ddf73 None None] MFA Rules not processed for user `7cf8cc6b2166466dbc8a886fb9c4043e`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:21.497665 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a1734e46-ffc2-419f-8221-4fe6134ddf73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:21.514000 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:21.567012 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1299/2613] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:14:21 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:14:21.578780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a7a8c-27a7-41e6-8b45-2747f6344d8e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:21.580773 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-803a7a8c-27a7-41e6-8b45-2747f6344d8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:21.637919 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a7a8c-27a7-41e6-8b45-2747f6344d8e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:21.642334 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76be63d-3304-4bc6-adcb-f5167f4a1c2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:21.642966 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76be63d-3304-4bc6-adcb-f5167f4a1c2f service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:21.643242 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76be63d-3304-4bc6-adcb-f5167f4a1c2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:21.643509 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76be63d-3304-4bc6-adcb-f5167f4a1c2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:21.668380 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76be63d-3304-4bc6-adcb-f5167f4a1c2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:21.668380 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76be63d-3304-4bc6-adcb-f5167f4a1c2f service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:21.668380 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1315/2614] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:14:21 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:14:21.817339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1734e46-ffc2-419f-8221-4fe6134ddf73 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:21.818738 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a1734e46-ffc2-419f-8221-4fe6134ddf73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:21.830832 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1734e46-ffc2-419f-8221-4fe6134ddf73 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:21.834993 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6604a0ec-ddbb-4081-824e-a5f2cebc630c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:21.835604 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604a0ec-ddbb-4081-824e-a5f2cebc630c service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:21.835866 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604a0ec-ddbb-4081-824e-a5f2cebc630c service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:21.836170 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604a0ec-ddbb-4081-824e-a5f2cebc630c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:21.891711 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6604a0ec-ddbb-4081-824e-a5f2cebc630c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:21.893188 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6604a0ec-ddbb-4081-824e-a5f2cebc630c service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:21.903183 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1300/2615] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:14:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:14:22.371958 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76be63d-3304-4bc6-adcb-f5167f4a1c2f service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:22.433423 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76be63d-3304-4bc6-adcb-f5167f4a1c2f service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:22.438093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0829b82-e3d6-412a-879a-017f6c0d91a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:22.438720 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0829b82-e3d6-412a-879a-017f6c0d91a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:22.439007 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0829b82-e3d6-412a-879a-017f6c0d91a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:22.439278 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0829b82-e3d6-412a-879a-017f6c0d91a6 admin admin] PATH_INFO: `/v3/users/c3cec4d5a20247b290cc86de6bbbfbb5` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:22.440462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0829b82-e3d6-412a-879a-017f6c0d91a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3cec4d5a20247b290cc86de6bbbfbb5)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:22.442390 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0829b82-e3d6-412a-879a-017f6c0d91a6 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:22.736378 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b0829b82-e3d6-412a-879a-017f6c0d91a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3cec4d5a20247b290cc86de6bbbfbb5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:22.737047 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b0829b82-e3d6-412a-879a-017f6c0d91a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3cec4d5a20247b290cc86de6bbbfbb5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:22.737661 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b0829b82-e3d6-412a-879a-017f6c0d91a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3cec4d5a20247b290cc86de6bbbfbb5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:22.744710 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b0829b82-e3d6-412a-879a-017f6c0d91a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3cec4d5a20247b290cc86de6bbbfbb5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:22.757533 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b0829b82-e3d6-412a-879a-017f6c0d91a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3cec4d5a20247b290cc86de6bbbfbb5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:22.842608 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1316/2616] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:14:22 2022] DELETE /identity/v3/users/c3cec4d5a20247b290cc86de6bbbfbb5 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:14:22.847271 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9173e-d368-4296-8c68-9c9e143f2db5 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:22.847529 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9173e-d368-4296-8c68-9c9e143f2db5 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:22.847782 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9173e-d368-4296-8c68-9c9e143f2db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:22.867857 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-ddf9173e-d368-4296-8c68-9c9e143f2db5 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:22.868519 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ddf9173e-d368-4296-8c68-9c9e143f2db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:22.927657 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1301/2617] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:14:22.937316 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0829b82-e3d6-412a-879a-017f6c0d91a6 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:22.938740 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b0829b82-e3d6-412a-879a-017f6c0d91a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:22.966046 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:22.996891 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0829b82-e3d6-412a-879a-017f6c0d91a6 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:23.000507 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581847b-8985-4899-a457-5192382189e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:23.007429 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581847b-8985-4899-a457-5192382189e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:23.007429 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581847b-8985-4899-a457-5192382189e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:23.007429 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581847b-8985-4899-a457-5192382189e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:23.011284 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1581847b-8985-4899-a457-5192382189e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:23.012872 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1581847b-8985-4899-a457-5192382189e9 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:23.023297 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1317/2618] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:14:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:14:23.383098 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9173e-d368-4296-8c68-9c9e143f2db5 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:23.384713 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ddf9173e-d368-4296-8c68-9c9e143f2db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:23.397955 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9173e-d368-4296-8c68-9c9e143f2db5 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:23.403029 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684cfee9-e5c2-42e8-80cc-c8237fa9fae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:23.403663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684cfee9-e5c2-42e8-80cc-c8237fa9fae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:23.404097 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684cfee9-e5c2-42e8-80cc-c8237fa9fae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:23.404481 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684cfee9-e5c2-42e8-80cc-c8237fa9fae9 admin admin] PATH_INFO: `/v3/projects/9e17598bfa9c440d9fed85f3ec3cffc1` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:23.405634 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684cfee9-e5c2-42e8-80cc-c8237fa9fae9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e17598bfa9c440d9fed85f3ec3cffc1)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:23.407544 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684cfee9-e5c2-42e8-80cc-c8237fa9fae9 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:23.512165 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:23.513749 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-684cfee9-e5c2-42e8-80cc-c8237fa9fae9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e17598bfa9c440d9fed85f3ec3cffc1', 'request_id': 'req-684cfee9-e5c2-42e8-80cc-c8237fa9fae9'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:23.574362 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-684cfee9-e5c2-42e8-80cc-c8237fa9fae9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e17598bfa9c440d9fed85f3ec3cffc1', 'request_id': 'req-684cfee9-e5c2-42e8-80cc-c8237fa9fae9'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:23.581850 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-684cfee9-e5c2-42e8-80cc-c8237fa9fae9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e17598bfa9c440d9fed85f3ec3cffc1', 'request_id': 'req-684cfee9-e5c2-42e8-80cc-c8237fa9fae9'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:23.592398 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-684cfee9-e5c2-42e8-80cc-c8237fa9fae9 admin admin] The token cache is being invalidate because project 9e17598bfa9c440d9fed85f3ec3cffc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:14:23.592694 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-684cfee9-e5c2-42e8-80cc-c8237fa9fae9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:23.593341 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-684cfee9-e5c2-42e8-80cc-c8237fa9fae9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:23.594854 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1302/2619] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:14:23 2022] DELETE /identity/v3/projects/9e17598bfa9c440d9fed85f3ec3cffc1 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:14:23.599492 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581847b-8985-4899-a457-5192382189e9 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:23.609101 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581847b-8985-4899-a457-5192382189e9 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:23.667920 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb2ea02-be39-4bc7-acfc-e9aadf56eae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:23.668504 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2ea02-be39-4bc7-acfc-e9aadf56eae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:23.673575 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2ea02-be39-4bc7-acfc-e9aadf56eae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:23.673575 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2ea02-be39-4bc7-acfc-e9aadf56eae8 admin admin] PATH_INFO: `/v3/users/7cf8cc6b2166466dbc8a886fb9c4043e` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:23.673575 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb2ea02-be39-4bc7-acfc-e9aadf56eae8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cf8cc6b2166466dbc8a886fb9c4043e)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:23.673575 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb2ea02-be39-4bc7-acfc-e9aadf56eae8 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:23.882877 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-3fb2ea02-be39-4bc7-acfc-e9aadf56eae8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cf8cc6b2166466dbc8a886fb9c4043e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:23.883618 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-3fb2ea02-be39-4bc7-acfc-e9aadf56eae8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cf8cc6b2166466dbc8a886fb9c4043e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:23.884258 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-3fb2ea02-be39-4bc7-acfc-e9aadf56eae8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cf8cc6b2166466dbc8a886fb9c4043e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:23.891362 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-3fb2ea02-be39-4bc7-acfc-e9aadf56eae8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cf8cc6b2166466dbc8a886fb9c4043e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:23.904188 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-3fb2ea02-be39-4bc7-acfc-e9aadf56eae8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cf8cc6b2166466dbc8a886fb9c4043e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:24.019927 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1318/2620] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:14:23 2022] DELETE /identity/v3/users/7cf8cc6b2166466dbc8a886fb9c4043e => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:14:24.387539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684cfee9-e5c2-42e8-80cc-c8237fa9fae9 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:24.389281 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-684cfee9-e5c2-42e8-80cc-c8237fa9fae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:24.404104 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684cfee9-e5c2-42e8-80cc-c8237fa9fae9 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:24.461488 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8013098a-5a9c-4cc1-9ca9-7fcc41448f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:24.462118 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013098a-5a9c-4cc1-9ca9-7fcc41448f1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:24.462395 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013098a-5a9c-4cc1-9ca9-7fcc41448f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:24.462667 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013098a-5a9c-4cc1-9ca9-7fcc41448f1f admin admin] PATH_INFO: `/v3/projects/7e68d76f0c0e4b14b02a968aa80ec188` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:24.463768 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8013098a-5a9c-4cc1-9ca9-7fcc41448f1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e68d76f0c0e4b14b02a968aa80ec188)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:24.465695 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8013098a-5a9c-4cc1-9ca9-7fcc41448f1f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:24.557324 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8013098a-5a9c-4cc1-9ca9-7fcc41448f1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e68d76f0c0e4b14b02a968aa80ec188', 'request_id': 'req-8013098a-5a9c-4cc1-9ca9-7fcc41448f1f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:24.619488 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8013098a-5a9c-4cc1-9ca9-7fcc41448f1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e68d76f0c0e4b14b02a968aa80ec188', 'request_id': 'req-8013098a-5a9c-4cc1-9ca9-7fcc41448f1f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:24.626165 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8013098a-5a9c-4cc1-9ca9-7fcc41448f1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e68d76f0c0e4b14b02a968aa80ec188', 'request_id': 'req-8013098a-5a9c-4cc1-9ca9-7fcc41448f1f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:24.637102 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8013098a-5a9c-4cc1-9ca9-7fcc41448f1f admin admin] The token cache is being invalidate because project 7e68d76f0c0e4b14b02a968aa80ec188 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:14:24.637391 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8013098a-5a9c-4cc1-9ca9-7fcc41448f1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:24.638025 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8013098a-5a9c-4cc1-9ca9-7fcc41448f1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:24.639495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1303/2621] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:14:24 2022] DELETE /identity/v3/projects/7e68d76f0c0e4b14b02a968aa80ec188 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:14:24.645198 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403059a-fc0f-48a0-8bd6-3ac95dfaf0c6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:24.645696 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403059a-fc0f-48a0-8bd6-3ac95dfaf0c6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:24.645961 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403059a-fc0f-48a0-8bd6-3ac95dfaf0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:24.668610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9403059a-fc0f-48a0-8bd6-3ac95dfaf0c6 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:24.669859 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9403059a-fc0f-48a0-8bd6-3ac95dfaf0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:24.713313 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:24.731566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1319/2622] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:14:24.739762 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418113c-9905-474f-84d5-1d04a36f2221 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:24.740056 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418113c-9905-474f-84d5-1d04a36f2221 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:24.740388 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418113c-9905-474f-84d5-1d04a36f2221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:24.760210 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-b418113c-9905-474f-84d5-1d04a36f2221 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:24.763955 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b418113c-9905-474f-84d5-1d04a36f2221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:24.769246 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1304/2623] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:14:24.773526 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9403059a-fc0f-48a0-8bd6-3ac95dfaf0c6 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:24.782885 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9403059a-fc0f-48a0-8bd6-3ac95dfaf0c6 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:24.788024 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d036eb8-d5d6-4f81-8eec-81a9f3a42fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:24.788643 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d036eb8-d5d6-4f81-8eec-81a9f3a42fef admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:24.788919 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d036eb8-d5d6-4f81-8eec-81a9f3a42fef admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:24.789188 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d036eb8-d5d6-4f81-8eec-81a9f3a42fef admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:24.789831 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d036eb8-d5d6-4f81-8eec-81a9f3a42fef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:14:24.790134 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d036eb8-d5d6-4f81-8eec-81a9f3a42fef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:24.791713 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d036eb8-d5d6-4f81-8eec-81a9f3a42fef admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:24.803541 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1320/2624] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:14:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:14:24.807663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418113c-9905-474f-84d5-1d04a36f2221 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:24.816778 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418113c-9905-474f-84d5-1d04a36f2221 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:24.821979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41077f8a-3f02-4665-bb7b-a2ae5cb9cfb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:24.822562 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41077f8a-3f02-4665-bb7b-a2ae5cb9cfb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:24.822839 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41077f8a-3f02-4665-bb7b-a2ae5cb9cfb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:24.823108 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41077f8a-3f02-4665-bb7b-a2ae5cb9cfb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:24.823740 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41077f8a-3f02-4665-bb7b-a2ae5cb9cfb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:24.825664 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41077f8a-3f02-4665-bb7b-a2ae5cb9cfb5 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:24.918859 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1305/2625] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:14:24 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:14:24.923042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d036eb8-d5d6-4f81-8eec-81a9f3a42fef admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:24.934602 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d036eb8-d5d6-4f81-8eec-81a9f3a42fef admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:25.003926 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af65b83e-8e44-4e75-a129-ff2ed7abfe9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:25.004571 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65b83e-8e44-4e75-a129-ff2ed7abfe9f admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:25.004847 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65b83e-8e44-4e75-a129-ff2ed7abfe9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:25.005171 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65b83e-8e44-4e75-a129-ff2ed7abfe9f admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:25.005802 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af65b83e-8e44-4e75-a129-ff2ed7abfe9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:25.007830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af65b83e-8e44-4e75-a129-ff2ed7abfe9f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:25.094859 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1321/2626] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:14:24 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:14:25.099539 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41077f8a-3f02-4665-bb7b-a2ae5cb9cfb5 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:25.109955 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41077f8a-3f02-4665-bb7b-a2ae5cb9cfb5 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:25.115193 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609f359-c659-4ceb-ab47-01961ed2565c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:25.115774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609f359-c659-4ceb-ab47-01961ed2565c admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:25.116047 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609f359-c659-4ceb-ab47-01961ed2565c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:25.116339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609f359-c659-4ceb-ab47-01961ed2565c admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:25.117096 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4609f359-c659-4ceb-ab47-01961ed2565c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:25.118652 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4609f359-c659-4ceb-ab47-01961ed2565c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:25.128740 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1306/2627] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:14:25.132562 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af65b83e-8e44-4e75-a129-ff2ed7abfe9f admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:25.142687 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af65b83e-8e44-4e75-a129-ff2ed7abfe9f admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:25.147340 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f9e4b-5ed2-4e51-8838-7801e3defcf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:25.147923 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f9e4b-5ed2-4e51-8838-7801e3defcf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:25.148234 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f9e4b-5ed2-4e51-8838-7801e3defcf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:25.148513 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f9e4b-5ed2-4e51-8838-7801e3defcf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:25.149155 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7f9e4b-5ed2-4e51-8838-7801e3defcf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:25.150739 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7f9e4b-5ed2-4e51-8838-7801e3defcf5 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:25.161652 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1322/2628] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:14:25.165590 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609f359-c659-4ceb-ab47-01961ed2565c admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:25.175430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609f359-c659-4ceb-ab47-01961ed2565c admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:25.180593 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b0cf95-589b-403a-9ab1-52ca23790ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:25.181228 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0cf95-589b-403a-9ab1-52ca23790ed4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:25.181504 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0cf95-589b-403a-9ab1-52ca23790ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:25.181772 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0cf95-589b-403a-9ab1-52ca23790ed4 admin admin] PATH_INFO: `/v3/projects/05a5096f0ff145bdacc4d391b5109110/users/e366443b10d0465fa454fcb019dfb065/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:25.196789 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b0cf95-589b-403a-9ab1-52ca23790ed4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05a5096f0ff145bdacc4d391b5109110, user_id=e366443b10d0465fa454fcb019dfb065, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:25.200722 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b0cf95-589b-403a-9ab1-52ca23790ed4 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:25.282547 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1307/2629] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:14:25 2022] PUT /identity/v3/projects/05a5096f0ff145bdacc4d391b5109110/users/e366443b10d0465fa454fcb019dfb065/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:14:25.294637 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80726c-bfeb-4e7a-b2a7-3fda64d9cd75 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:25.295497 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80726c-bfeb-4e7a-b2a7-3fda64d9cd75 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:25.295497 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80726c-bfeb-4e7a-b2a7-3fda64d9cd75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:25.329092 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ea80726c-bfeb-4e7a-b2a7-3fda64d9cd75 None None] MFA Rules not processed for user `e366443b10d0465fa454fcb019dfb065`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:25.329092 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ea80726c-bfeb-4e7a-b2a7-3fda64d9cd75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:25.388298 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1323/2630] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:14:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:14:25.399939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b0cf95-589b-403a-9ab1-52ca23790ed4 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:25.401668 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-92b0cf95-589b-403a-9ab1-52ca23790ed4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:25.406880 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:25.459559 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b0cf95-589b-403a-9ab1-52ca23790ed4 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:25.463256 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb30e3e-5b34-4704-b90b-2e08cabaa42a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7832e97d7e97439e821268d05816ae3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:25.463933 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb30e3e-5b34-4704-b90b-2e08cabaa42a service glance] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:25.464418 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb30e3e-5b34-4704-b90b-2e08cabaa42a service glance] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:25.464911 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb30e3e-5b34-4704-b90b-2e08cabaa42a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:25.474263 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb30e3e-5b34-4704-b90b-2e08cabaa42a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:25.478751 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb30e3e-5b34-4704-b90b-2e08cabaa42a service glance] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:25.484472 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1308/2631] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:14:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:14:30.973857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea80726c-bfeb-4e7a-b2a7-3fda64d9cd75 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:30.975572 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ea80726c-bfeb-4e7a-b2a7-3fda64d9cd75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:31.019138 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:31.045574 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea80726c-bfeb-4e7a-b2a7-3fda64d9cd75 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:31.050666 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc898ce-21fb-47b1-af89-1fed689e0a28 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4f6de2065d4fbea3fa141da760f708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:31.051298 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc898ce-21fb-47b1-af89-1fed689e0a28 service placement] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:31.051483 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc898ce-21fb-47b1-af89-1fed689e0a28 service placement] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:31.051764 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc898ce-21fb-47b1-af89-1fed689e0a28 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:31.054639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-dbc898ce-21fb-47b1-af89-1fed689e0a28 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:31.066702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc898ce-21fb-47b1-af89-1fed689e0a28 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:31.068857 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc898ce-21fb-47b1-af89-1fed689e0a28 service placement] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:31.078527 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1324/2632] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:14:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:14:35.480501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=25 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:41.088214 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=14 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:44.789457 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d980ab5-de53-4251-8d3c-1eedd7faa8a2 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:44.789979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d980ab5-de53-4251-8d3c-1eedd7faa8a2 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:44.789979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d980ab5-de53-4251-8d3c-1eedd7faa8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:44.818997 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-2d980ab5-de53-4251-8d3c-1eedd7faa8a2 None None] MFA Rules not processed for user `8c21ea7d3cbb494886e60a4bd4f39145`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:44.820265 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2d980ab5-de53-4251-8d3c-1eedd7faa8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:44.880998 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1309/2633] 10.222.0.48 () {58 vars in 1009 bytes} [Fri Sep 9 21:14:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:14:45.024280 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc898ce-21fb-47b1-af89-1fed689e0a28 service placement] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:45.026027 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-dbc898ce-21fb-47b1-af89-1fed689e0a28 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:45.102261 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc898ce-21fb-47b1-af89-1fed689e0a28 service placement] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:45.107385 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc0fdf-d2f9-4c0d-9b2e-7fcd698e7c18 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d28fa5d49d44120bd5eaec2d4e5fdbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:45.107995 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc0fdf-d2f9-4c0d-9b2e-7fcd698e7c18 service swift] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:45.108331 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc0fdf-d2f9-4c0d-9b2e-7fcd698e7c18 service swift] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:45.108609 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc0fdf-d2f9-4c0d-9b2e-7fcd698e7c18 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:45.116959 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc0fdf-d2f9-4c0d-9b2e-7fcd698e7c18 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:45.118548 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc0fdf-d2f9-4c0d-9b2e-7fcd698e7c18 service swift] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:45.126572 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1325/2634] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Sep 9 21:14:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 21:14:52.426447 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d980ab5-de53-4251-8d3c-1eedd7faa8a2 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:52.519981 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d980ab5-de53-4251-8d3c-1eedd7faa8a2 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:52.528945 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e91ca-242d-4c33-a5df-afabe49453f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:52.529517 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e91ca-242d-4c33-a5df-afabe49453f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:52.529797 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e91ca-242d-4c33-a5df-afabe49453f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:52.530073 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e91ca-242d-4c33-a5df-afabe49453f9 admin admin] PATH_INFO: `/v3/users/e366443b10d0465fa454fcb019dfb065` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:52.531232 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5e91ca-242d-4c33-a5df-afabe49453f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e366443b10d0465fa454fcb019dfb065)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:52.533304 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5e91ca-242d-4c33-a5df-afabe49453f9 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:52.733847 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-df5e91ca-242d-4c33-a5df-afabe49453f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e366443b10d0465fa454fcb019dfb065'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:52.734538 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-df5e91ca-242d-4c33-a5df-afabe49453f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e366443b10d0465fa454fcb019dfb065'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:52.745833 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-df5e91ca-242d-4c33-a5df-afabe49453f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e366443b10d0465fa454fcb019dfb065'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:52.760686 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-df5e91ca-242d-4c33-a5df-afabe49453f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e366443b10d0465fa454fcb019dfb065'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:52.762086 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-df5e91ca-242d-4c33-a5df-afabe49453f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e366443b10d0465fa454fcb019dfb065'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:52.837758 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1310/2635] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:14:52 2022] DELETE /identity/v3/users/e366443b10d0465fa454fcb019dfb065 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:14:52.842644 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d718ff4-b9ce-4ef4-9d47-0bebb6f4dcf8 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:52.843136 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d718ff4-b9ce-4ef4-9d47-0bebb6f4dcf8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:52.843234 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d718ff4-b9ce-4ef4-9d47-0bebb6f4dcf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:52.863208 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5d718ff4-b9ce-4ef4-9d47-0bebb6f4dcf8 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:52.864554 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5d718ff4-b9ce-4ef4-9d47-0bebb6f4dcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:52.934866 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1326/2636] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:14:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:14:52.945715 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e91ca-242d-4c33-a5df-afabe49453f9 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:52.948134 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-df5e91ca-242d-4c33-a5df-afabe49453f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:52.964718 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:53.053475 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e91ca-242d-4c33-a5df-afabe49453f9 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:53.060205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a61539-4455-4426-8727-1c119441a3de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:53.064432 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a61539-4455-4426-8727-1c119441a3de service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:53.064432 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a61539-4455-4426-8727-1c119441a3de service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:53.064432 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a61539-4455-4426-8727-1c119441a3de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:53.074759 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a61539-4455-4426-8727-1c119441a3de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:53.076417 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a61539-4455-4426-8727-1c119441a3de service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:53.087914 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1311/2637] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:14:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:14:53.638107 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d718ff4-b9ce-4ef4-9d47-0bebb6f4dcf8 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:53.639752 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5d718ff4-b9ce-4ef4-9d47-0bebb6f4dcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:53.655569 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d718ff4-b9ce-4ef4-9d47-0bebb6f4dcf8 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:53.662231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247ef95-2bf0-4417-a1a7-0a515539c28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:53.663861 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247ef95-2bf0-4417-a1a7-0a515539c28d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:53.664596 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247ef95-2bf0-4417-a1a7-0a515539c28d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:53.665069 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247ef95-2bf0-4417-a1a7-0a515539c28d admin admin] PATH_INFO: `/v3/projects/05a5096f0ff145bdacc4d391b5109110` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:53.666515 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247ef95-2bf0-4417-a1a7-0a515539c28d admin admin] RBAC: Authorizing `identity:delete_project(project_id=05a5096f0ff145bdacc4d391b5109110)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:53.670145 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247ef95-2bf0-4417-a1a7-0a515539c28d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:53.825504 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2247ef95-2bf0-4417-a1a7-0a515539c28d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05a5096f0ff145bdacc4d391b5109110', 'request_id': 'req-2247ef95-2bf0-4417-a1a7-0a515539c28d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:53.833912 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2247ef95-2bf0-4417-a1a7-0a515539c28d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05a5096f0ff145bdacc4d391b5109110', 'request_id': 'req-2247ef95-2bf0-4417-a1a7-0a515539c28d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:53.927900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2247ef95-2bf0-4417-a1a7-0a515539c28d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05a5096f0ff145bdacc4d391b5109110', 'request_id': 'req-2247ef95-2bf0-4417-a1a7-0a515539c28d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:53.942126 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2247ef95-2bf0-4417-a1a7-0a515539c28d admin admin] The token cache is being invalidate because project 05a5096f0ff145bdacc4d391b5109110 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:14:53.942126 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2247ef95-2bf0-4417-a1a7-0a515539c28d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:53.942898 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2247ef95-2bf0-4417-a1a7-0a515539c28d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:14:53.945068 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1327/2638] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:14:53 2022] DELETE /identity/v3/projects/05a5096f0ff145bdacc4d391b5109110 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:14:53.950733 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d647d-b0cb-4eec-b927-c0e2d29c5f3d None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:53.951230 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d647d-b0cb-4eec-b927-c0e2d29c5f3d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:53.951340 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d647d-b0cb-4eec-b927-c0e2d29c5f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:53.975717 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-291d647d-b0cb-4eec-b927-c0e2d29c5f3d None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:53.976989 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-291d647d-b0cb-4eec-b927-c0e2d29c5f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:54.053315 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1312/2639] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:14:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:14:54.061162 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d211b0e-2828-4e91-92ca-532f11783e21 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:54.061708 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d211b0e-2828-4e91-92ca-532f11783e21 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:54.062097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d211b0e-2828-4e91-92ca-532f11783e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:54.084414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-6d211b0e-2828-4e91-92ca-532f11783e21 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:54.085839 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6d211b0e-2828-4e91-92ca-532f11783e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:54.094914 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1328/2640] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:14:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:14:54.098848 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d647d-b0cb-4eec-b927-c0e2d29c5f3d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:54.110988 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d647d-b0cb-4eec-b927-c0e2d29c5f3d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:54.117186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7677c205-8f8f-4101-97da-947dbfe6ca7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:54.117706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677c205-8f8f-4101-97da-947dbfe6ca7b admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:54.118007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677c205-8f8f-4101-97da-947dbfe6ca7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:54.118274 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677c205-8f8f-4101-97da-947dbfe6ca7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:54.118921 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7677c205-8f8f-4101-97da-947dbfe6ca7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:14:54.119222 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7677c205-8f8f-4101-97da-947dbfe6ca7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:54.120860 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7677c205-8f8f-4101-97da-947dbfe6ca7b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:54.133069 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1313/2641] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:14:54.137957 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d211b0e-2828-4e91-92ca-532f11783e21 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:54.147398 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d211b0e-2828-4e91-92ca-532f11783e21 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:54.156031 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a073d07-577c-4bd8-a111-93b77b1a0935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:54.156749 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a073d07-577c-4bd8-a111-93b77b1a0935 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:54.157088 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a073d07-577c-4bd8-a111-93b77b1a0935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:54.157444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a073d07-577c-4bd8-a111-93b77b1a0935 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:54.158148 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a073d07-577c-4bd8-a111-93b77b1a0935 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:54.161410 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a073d07-577c-4bd8-a111-93b77b1a0935 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:54.169988 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:54.255284 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1329/2642] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:14:54 2022] POST /identity/v3/projects => generated 377 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:14:54.259648 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7677c205-8f8f-4101-97da-947dbfe6ca7b admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:54.271290 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7677c205-8f8f-4101-97da-947dbfe6ca7b admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:54.339476 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9fae76-43fa-45e8-8655-397ed48669ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:54.340157 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fae76-43fa-45e8-8655-397ed48669ae admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:54.340374 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fae76-43fa-45e8-8655-397ed48669ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:54.340647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fae76-43fa-45e8-8655-397ed48669ae admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:54.341281 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9fae76-43fa-45e8-8655-397ed48669ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:54.343181 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9fae76-43fa-45e8-8655-397ed48669ae admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:54.791613 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1314/2643] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:14:54 2022] POST /identity/v3/users => generated 289 bytes in 532 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:14:54.797165 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a073d07-577c-4bd8-a111-93b77b1a0935 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:54.808686 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a073d07-577c-4bd8-a111-93b77b1a0935 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:54.813423 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1897a335-e9a9-4d43-a305-fc45e0f4c6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:54.814069 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1897a335-e9a9-4d43-a305-fc45e0f4c6a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:54.814349 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1897a335-e9a9-4d43-a305-fc45e0f4c6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:54.814623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1897a335-e9a9-4d43-a305-fc45e0f4c6a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:54.815280 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1897a335-e9a9-4d43-a305-fc45e0f4c6a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:54.816940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1897a335-e9a9-4d43-a305-fc45e0f4c6a7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:54.831309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1330/2644] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:14:54.834444 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9fae76-43fa-45e8-8655-397ed48669ae admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:54.842954 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9fae76-43fa-45e8-8655-397ed48669ae admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:54.847718 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9c81d-e93b-4e0d-9d0e-713513d9eab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:54.848372 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9c81d-e93b-4e0d-9d0e-713513d9eab6 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:54.848656 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9c81d-e93b-4e0d-9d0e-713513d9eab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:54.848932 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9c81d-e93b-4e0d-9d0e-713513d9eab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:54.849590 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9c81d-e93b-4e0d-9d0e-713513d9eab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:54.851198 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9c81d-e93b-4e0d-9d0e-713513d9eab6 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:54.866573 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1315/2645] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:14:54.867774 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1897a335-e9a9-4d43-a305-fc45e0f4c6a7 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:54.878797 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1897a335-e9a9-4d43-a305-fc45e0f4c6a7 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:54.883740 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b5e45f-ac5b-4574-9213-2abec3fd79ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:54.885716 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5e45f-ac5b-4574-9213-2abec3fd79ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:54.885716 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5e45f-ac5b-4574-9213-2abec3fd79ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:54.885716 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5e45f-ac5b-4574-9213-2abec3fd79ef admin admin] PATH_INFO: `/v3/projects/facf2910c1fe4061bb38ecd59c9a70c6/users/f5ee4306e4334a9fb9f61025ed5fb436/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:54.901267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b5e45f-ac5b-4574-9213-2abec3fd79ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=facf2910c1fe4061bb38ecd59c9a70c6, user_id=f5ee4306e4334a9fb9f61025ed5fb436, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:54.904105 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b5e45f-ac5b-4574-9213-2abec3fd79ef admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:55.021681 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1331/2646] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:14:54 2022] PUT /identity/v3/projects/facf2910c1fe4061bb38ecd59c9a70c6/users/f5ee4306e4334a9fb9f61025ed5fb436/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:14:55.037200 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6afd4b-fbec-42ae-9f5f-27df07d2894c None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:55.037779 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6afd4b-fbec-42ae-9f5f-27df07d2894c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:55.037779 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6afd4b-fbec-42ae-9f5f-27df07d2894c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:55.073832 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-4f6afd4b-fbec-42ae-9f5f-27df07d2894c None None] MFA Rules not processed for user `f5ee4306e4334a9fb9f61025ed5fb436`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:55.075249 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-4f6afd4b-fbec-42ae-9f5f-27df07d2894c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:55.081600 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:55.162403 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1316/2647] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:14:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:14:55.175328 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b5e45f-ac5b-4574-9213-2abec3fd79ef admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:55.177056 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-77b5e45f-ac5b-4574-9213-2abec3fd79ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:55.255592 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b5e45f-ac5b-4574-9213-2abec3fd79ef admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:55.260002 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26dc78f-2980-4ae2-a651-9262b9c254bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:55.260679 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26dc78f-2980-4ae2-a651-9262b9c254bb service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:55.261642 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26dc78f-2980-4ae2-a651-9262b9c254bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:55.261919 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26dc78f-2980-4ae2-a651-9262b9c254bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:55.283847 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26dc78f-2980-4ae2-a651-9262b9c254bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:55.285548 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26dc78f-2980-4ae2-a651-9262b9c254bb service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:55.296752 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1332/2648] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:14:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:15:05.292673 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=47,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:05.408263 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=24 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:12.888712 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6afd4b-fbec-42ae-9f5f-27df07d2894c None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:12.890172 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-4f6afd4b-fbec-42ae-9f5f-27df07d2894c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:12.950885 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6afd4b-fbec-42ae-9f5f-27df07d2894c None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:12.954634 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bb8da8-0066-4768-91f1-e96c45001cfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:12.955619 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb8da8-0066-4768-91f1-e96c45001cfb service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:12.955963 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb8da8-0066-4768-91f1-e96c45001cfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:12.956329 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb8da8-0066-4768-91f1-e96c45001cfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:12.958125 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a7bb8da8-0066-4768-91f1-e96c45001cfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:13.032104 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bb8da8-0066-4768-91f1-e96c45001cfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:13.033611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bb8da8-0066-4768-91f1-e96c45001cfb service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:13.044288 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1317/2649] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:15:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:15:15.056036 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d1847-192a-4a38-a917-ed6b78563b67 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:15.056620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d1847-192a-4a38-a917-ed6b78563b67 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:15.056620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d1847-192a-4a38-a917-ed6b78563b67 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:15.058640 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1333/2650] 10.222.0.48 () {54 vars in 921 bytes} [Fri Sep 9 21:15:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:15:15.064617 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a2483-381f-4139-b575-3db31dd54765 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:15.065101 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a2483-381f-4139-b575-3db31dd54765 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:15.065394 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a2483-381f-4139-b575-3db31dd54765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:15.143754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-8a0a2483-381f-4139-b575-3db31dd54765 None None] MFA Rules not processed for user `613b845a9b4d432daf8d478fc3970708`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:15:15.145108 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8a0a2483-381f-4139-b575-3db31dd54765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:15.242491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1318/2651] 10.222.0.48 () {58 vars in 1018 bytes} [Fri Sep 9 21:15:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:15:15.250476 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d1847-192a-4a38-a917-ed6b78563b67 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:15.252413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-dd9d1847-192a-4a38-a917-ed6b78563b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:15.326810 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d1847-192a-4a38-a917-ed6b78563b67 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:15.330707 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f4e76-d7e0-4066-b5f2-af510ae9454c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7832e97d7e97439e821268d05816ae3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:15.331477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f4e76-d7e0-4066-b5f2-af510ae9454c service glance] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:15.331832 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f4e76-d7e0-4066-b5f2-af510ae9454c service glance] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:15.332218 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f4e76-d7e0-4066-b5f2-af510ae9454c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:15.342567 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f4e76-d7e0-4066-b5f2-af510ae9454c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:15.344129 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f4e76-d7e0-4066-b5f2-af510ae9454c service glance] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:15.354803 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1334/2652] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:15:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:15:15.604399 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239cff6e-e8cd-44a1-8955-f9cf6969f54e None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:15.605028 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239cff6e-e8cd-44a1-8955-f9cf6969f54e None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:15.605522 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239cff6e-e8cd-44a1-8955-f9cf6969f54e None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:15.607327 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1319/2653] 10.222.0.48 () {54 vars in 921 bytes} [Fri Sep 9 21:15:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:15:15.612929 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217887d-474a-481b-a959-8b15f1a1c880 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:15.613510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217887d-474a-481b-a959-8b15f1a1c880 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:15.613791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217887d-474a-481b-a959-8b15f1a1c880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:15.640290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e217887d-474a-481b-a959-8b15f1a1c880 None None] MFA Rules not processed for user `8f5dfac06ac9492a86503ba133c638a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:15:15.641550 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e217887d-474a-481b-a959-8b15f1a1c880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:15.648270 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1335/2654] 10.222.0.48 () {58 vars in 1018 bytes} [Fri Sep 9 21:15:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:15:15.660288 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239cff6e-e8cd-44a1-8955-f9cf6969f54e None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:15.662022 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-239cff6e-e8cd-44a1-8955-f9cf6969f54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:15.677079 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239cff6e-e8cd-44a1-8955-f9cf6969f54e None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:15.681080 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264cc2df-2006-4fbb-a3ea-d85d3337637c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:15.681805 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cc2df-2006-4fbb-a3ea-d85d3337637c service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:15.682176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cc2df-2006-4fbb-a3ea-d85d3337637c service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:15.682544 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cc2df-2006-4fbb-a3ea-d85d3337637c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:15.691854 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264cc2df-2006-4fbb-a3ea-d85d3337637c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:15.693382 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264cc2df-2006-4fbb-a3ea-d85d3337637c service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:15.702103 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1320/2655] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:15:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:15:15.796797 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788313b-da6b-4043-927a-b7ef98322b73 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:15.797081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788313b-da6b-4043-927a-b7ef98322b73 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:15.797353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788313b-da6b-4043-927a-b7ef98322b73 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:15.798687 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1336/2656] 10.222.0.48 () {54 vars in 921 bytes} [Fri Sep 9 21:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:15:15.804156 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb20082-626f-45cc-b428-dc4f928292c2 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:15.804464 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb20082-626f-45cc-b428-dc4f928292c2 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:15.804737 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb20082-626f-45cc-b428-dc4f928292c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:15.825622 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-edb20082-626f-45cc-b428-dc4f928292c2 None None] MFA Rules not processed for user `613b845a9b4d432daf8d478fc3970708`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:15:15.826836 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-edb20082-626f-45cc-b428-dc4f928292c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:15.838598 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1321/2657] 10.222.0.48 () {58 vars in 1018 bytes} [Fri Sep 9 21:15:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:15:15.847816 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788313b-da6b-4043-927a-b7ef98322b73 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:15.859512 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788313b-da6b-4043-927a-b7ef98322b73 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:15.863289 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a190f17-c08d-4496-943e-e610b452fda8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:15.863884 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a190f17-c08d-4496-943e-e610b452fda8 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:15.864173 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a190f17-c08d-4496-943e-e610b452fda8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:15.864436 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a190f17-c08d-4496-943e-e610b452fda8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:15.877115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a190f17-c08d-4496-943e-e610b452fda8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:15.878586 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a190f17-c08d-4496-943e-e610b452fda8 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:15.890247 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1337/2658] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:15:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:15:18.841505 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb20082-626f-45cc-b428-dc4f928292c2 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:18.846000 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:18.852062 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb20082-626f-45cc-b428-dc4f928292c2 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:18.863135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287621c7-bf70-4156-afc8-107223f71438 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:18.863135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287621c7-bf70-4156-afc8-107223f71438 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:18.863135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287621c7-bf70-4156-afc8-107223f71438 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:18.863135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287621c7-bf70-4156-afc8-107223f71438 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:18.863509 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-287621c7-bf70-4156-afc8-107223f71438 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:18.877084 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287621c7-bf70-4156-afc8-107223f71438 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:18.878702 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287621c7-bf70-4156-afc8-107223f71438 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:18.890662 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1322/2659] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:15:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:15:20.115727 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a190f17-c08d-4496-943e-e610b452fda8 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:20.126123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a190f17-c08d-4496-943e-e610b452fda8 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:20.130008 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c09186-1d8e-483f-8f3c-8d2c1ccaef7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:20.130598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c09186-1d8e-483f-8f3c-8d2c1ccaef7f service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:20.131040 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c09186-1d8e-483f-8f3c-8d2c1ccaef7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:20.131311 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c09186-1d8e-483f-8f3c-8d2c1ccaef7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:20.133081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-58c09186-1d8e-483f-8f3c-8d2c1ccaef7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:20.149491 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c09186-1d8e-483f-8f3c-8d2c1ccaef7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:20.150971 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c09186-1d8e-483f-8f3c-8d2c1ccaef7f service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:20.162632 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1338/2660] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:15:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:15:20.640443 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287621c7-bf70-4156-afc8-107223f71438 service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:20.642029 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-287621c7-bf70-4156-afc8-107223f71438 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:20.668755 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287621c7-bf70-4156-afc8-107223f71438 service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:20.671427 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f368e-4d76-4fee-958b-9995b2249ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:20.672197 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f368e-4d76-4fee-958b-9995b2249ed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:20.672550 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f368e-4d76-4fee-958b-9995b2249ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:20.672921 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f368e-4d76-4fee-958b-9995b2249ed6 admin admin] PATH_INFO: `/v3/users/b17adcb2be25427485d5004c7821e094` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:20.674185 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f368e-4d76-4fee-958b-9995b2249ed6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b17adcb2be25427485d5004c7821e094)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:20.676321 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f368e-4d76-4fee-958b-9995b2249ed6 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:20.874908 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-890f368e-4d76-4fee-958b-9995b2249ed6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b17adcb2be25427485d5004c7821e094'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:15:20.877103 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-890f368e-4d76-4fee-958b-9995b2249ed6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b17adcb2be25427485d5004c7821e094'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:15:20.884618 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-890f368e-4d76-4fee-958b-9995b2249ed6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b17adcb2be25427485d5004c7821e094'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:15:20.896651 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-890f368e-4d76-4fee-958b-9995b2249ed6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b17adcb2be25427485d5004c7821e094'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:15:20.897381 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-890f368e-4d76-4fee-958b-9995b2249ed6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b17adcb2be25427485d5004c7821e094'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:15:20.974149 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1323/2661] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:15:20 2022] DELETE /identity/v3/users/b17adcb2be25427485d5004c7821e094 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:15:21.159935 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d218c8e-c2bb-4ec3-be15-b67b241a085f None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:21.166465 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d218c8e-c2bb-4ec3-be15-b67b241a085f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:21.166465 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d218c8e-c2bb-4ec3-be15-b67b241a085f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:21.179134 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1d218c8e-c2bb-4ec3-be15-b67b241a085f None None] MFA Rules not processed for user `8c21ea7d3cbb494886e60a4bd4f39145`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:15:21.180365 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1d218c8e-c2bb-4ec3-be15-b67b241a085f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:21.237900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1339/2662] 10.222.0.48 () {58 vars in 1009 bytes} [Fri Sep 9 21:15:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:15:21.238089 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f368e-4d76-4fee-958b-9995b2249ed6 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:21.239515 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-890f368e-4d76-4fee-958b-9995b2249ed6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:21.251204 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d218c8e-c2bb-4ec3-be15-b67b241a085f None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:21.253202 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1d218c8e-c2bb-4ec3-be15-b67b241a085f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:21.314646 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f368e-4d76-4fee-958b-9995b2249ed6 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:21.319481 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9a124-66b0-4d9e-b79d-5e4ac03b4095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:21.321392 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9a124-66b0-4d9e-b79d-5e4ac03b4095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:21.321392 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9a124-66b0-4d9e-b79d-5e4ac03b4095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:21.321392 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9a124-66b0-4d9e-b79d-5e4ac03b4095 admin admin] PATH_INFO: `/v3/projects/d26a3ee560e449b8b1188b9defa98fa6` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:21.322412 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f9a124-66b0-4d9e-b79d-5e4ac03b4095 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d26a3ee560e449b8b1188b9defa98fa6)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:21.324936 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f9a124-66b0-4d9e-b79d-5e4ac03b4095 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:21.331695 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d218c8e-c2bb-4ec3-be15-b67b241a085f None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:21.336939 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdecee7-8751-4a16-9c98-4aba3c9b82ff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d28fa5d49d44120bd5eaec2d4e5fdbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:21.338067 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdecee7-8751-4a16-9c98-4aba3c9b82ff service swift] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:21.338272 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdecee7-8751-4a16-9c98-4aba3c9b82ff service swift] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:21.338550 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdecee7-8751-4a16-9c98-4aba3c9b82ff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:21.346914 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdecee7-8751-4a16-9c98-4aba3c9b82ff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:21.348568 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdecee7-8751-4a16-9c98-4aba3c9b82ff service swift] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:21.356903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1340/2663] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Sep 9 21:15:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 21:15:21.419505 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a7f9a124-66b0-4d9e-b79d-5e4ac03b4095 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd26a3ee560e449b8b1188b9defa98fa6', 'request_id': 'req-a7f9a124-66b0-4d9e-b79d-5e4ac03b4095'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:15:21.477110 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a7f9a124-66b0-4d9e-b79d-5e4ac03b4095 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd26a3ee560e449b8b1188b9defa98fa6', 'request_id': 'req-a7f9a124-66b0-4d9e-b79d-5e4ac03b4095'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:15:21.490824 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a7f9a124-66b0-4d9e-b79d-5e4ac03b4095 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd26a3ee560e449b8b1188b9defa98fa6', 'request_id': 'req-a7f9a124-66b0-4d9e-b79d-5e4ac03b4095'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:15:21.493751 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:21.508293 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a7f9a124-66b0-4d9e-b79d-5e4ac03b4095 admin admin] The token cache is being invalidate because project d26a3ee560e449b8b1188b9defa98fa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:15:21.508627 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a7f9a124-66b0-4d9e-b79d-5e4ac03b4095 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:15:21.509284 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a7f9a124-66b0-4d9e-b79d-5e4ac03b4095 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:15:21.516448 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1324/2664] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:15:21 2022] DELETE /identity/v3/projects/d26a3ee560e449b8b1188b9defa98fa6 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:15:21.520333 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdecee7-8751-4a16-9c98-4aba3c9b82ff service swift] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:21.532162 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:21.537468 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdecee7-8751-4a16-9c98-4aba3c9b82ff service swift] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:21.637439 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe2fd8-caad-42d6-80ea-bf4846ee6b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:21.638687 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe2fd8-caad-42d6-80ea-bf4846ee6b95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:21.639311 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe2fd8-caad-42d6-80ea-bf4846ee6b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:21.640003 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe2fd8-caad-42d6-80ea-bf4846ee6b95 admin admin] PATH_INFO: `/v3/users/6bcc5313cdc04538a6b0f344cd369aab` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:21.641944 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fe2fd8-caad-42d6-80ea-bf4846ee6b95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bcc5313cdc04538a6b0f344cd369aab)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:21.644910 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fe2fd8-caad-42d6-80ea-bf4846ee6b95 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:21.854433 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b7fe2fd8-caad-42d6-80ea-bf4846ee6b95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bcc5313cdc04538a6b0f344cd369aab'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:15:21.855572 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b7fe2fd8-caad-42d6-80ea-bf4846ee6b95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bcc5313cdc04538a6b0f344cd369aab'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:15:21.856275 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b7fe2fd8-caad-42d6-80ea-bf4846ee6b95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bcc5313cdc04538a6b0f344cd369aab'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:15:21.864186 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b7fe2fd8-caad-42d6-80ea-bf4846ee6b95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bcc5313cdc04538a6b0f344cd369aab'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:15:21.879774 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b7fe2fd8-caad-42d6-80ea-bf4846ee6b95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bcc5313cdc04538a6b0f344cd369aab'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:15:21.953588 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1341/2665] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:15:21 2022] DELETE /identity/v3/users/6bcc5313cdc04538a6b0f344cd369aab => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:15:22.302343 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9a124-66b0-4d9e-b79d-5e4ac03b4095 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:22.303939 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a7f9a124-66b0-4d9e-b79d-5e4ac03b4095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:22.326820 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9a124-66b0-4d9e-b79d-5e4ac03b4095 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:22.406038 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38249737-1828-4ae0-b79f-758f82658dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:22.407209 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38249737-1828-4ae0-b79f-758f82658dac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:22.407760 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38249737-1828-4ae0-b79f-758f82658dac admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:22.408357 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38249737-1828-4ae0-b79f-758f82658dac admin admin] PATH_INFO: `/v3/projects/c51c747a278c42608b5091d206ff29cc` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:22.409930 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38249737-1828-4ae0-b79f-758f82658dac admin admin] RBAC: Authorizing `identity:delete_project(project_id=c51c747a278c42608b5091d206ff29cc)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:22.412280 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38249737-1828-4ae0-b79f-758f82658dac admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:22.546551 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-38249737-1828-4ae0-b79f-758f82658dac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c51c747a278c42608b5091d206ff29cc', 'request_id': 'req-38249737-1828-4ae0-b79f-758f82658dac'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:15:22.641826 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-38249737-1828-4ae0-b79f-758f82658dac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c51c747a278c42608b5091d206ff29cc', 'request_id': 'req-38249737-1828-4ae0-b79f-758f82658dac'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:15:22.651073 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-38249737-1828-4ae0-b79f-758f82658dac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c51c747a278c42608b5091d206ff29cc', 'request_id': 'req-38249737-1828-4ae0-b79f-758f82658dac'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:15:22.665317 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-38249737-1828-4ae0-b79f-758f82658dac admin admin] The token cache is being invalidate because project c51c747a278c42608b5091d206ff29cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:15:22.665317 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-38249737-1828-4ae0-b79f-758f82658dac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:15:22.665317 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-38249737-1828-4ae0-b79f-758f82658dac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:15:22.676106 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1325/2666] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:15:22 2022] DELETE /identity/v3/projects/c51c747a278c42608b5091d206ff29cc => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:15:26.148175 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe2fd8-caad-42d6-80ea-bf4846ee6b95 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:26.149593 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b7fe2fd8-caad-42d6-80ea-bf4846ee6b95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:26.163541 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe2fd8-caad-42d6-80ea-bf4846ee6b95 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:26.227584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eae23c-ad28-450e-b4e5-8f1f943f703a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:26.228270 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eae23c-ad28-450e-b4e5-8f1f943f703a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:26.228638 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eae23c-ad28-450e-b4e5-8f1f943f703a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:26.229130 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eae23c-ad28-450e-b4e5-8f1f943f703a admin admin] PATH_INFO: `/v3/users/f5ee4306e4334a9fb9f61025ed5fb436` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:26.231302 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eae23c-ad28-450e-b4e5-8f1f943f703a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5ee4306e4334a9fb9f61025ed5fb436)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:26.233818 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eae23c-ad28-450e-b4e5-8f1f943f703a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:26.340115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:26.428953 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f1eae23c-ad28-450e-b4e5-8f1f943f703a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5ee4306e4334a9fb9f61025ed5fb436'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:15:26.430665 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f1eae23c-ad28-450e-b4e5-8f1f943f703a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5ee4306e4334a9fb9f61025ed5fb436'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:15:26.431290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f1eae23c-ad28-450e-b4e5-8f1f943f703a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5ee4306e4334a9fb9f61025ed5fb436'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:15:26.439176 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f1eae23c-ad28-450e-b4e5-8f1f943f703a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5ee4306e4334a9fb9f61025ed5fb436'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:15:26.452786 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f1eae23c-ad28-450e-b4e5-8f1f943f703a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5ee4306e4334a9fb9f61025ed5fb436'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:15:26.516632 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1342/2667] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:15:26 2022] DELETE /identity/v3/users/f5ee4306e4334a9fb9f61025ed5fb436 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:15:26.521133 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1d21a-a9dc-4f68-ae10-2858dd4cf5cf None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:26.521696 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1d21a-a9dc-4f68-ae10-2858dd4cf5cf None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:26.521821 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1d21a-a9dc-4f68-ae10-2858dd4cf5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:26.547651 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-32f1d21a-a9dc-4f68-ae10-2858dd4cf5cf None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:15:26.549018 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-32f1d21a-a9dc-4f68-ae10-2858dd4cf5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:26.630339 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1326/2668] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:15:26.640478 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eae23c-ad28-450e-b4e5-8f1f943f703a admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:26.641913 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f1eae23c-ad28-450e-b4e5-8f1f943f703a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:26.711326 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eae23c-ad28-450e-b4e5-8f1f943f703a admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:26.715444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ca43f-2cd0-4524-a2b2-bc2f8b33c701 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:26.716061 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ca43f-2cd0-4524-a2b2-bc2f8b33c701 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:26.716366 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ca43f-2cd0-4524-a2b2-bc2f8b33c701 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:26.716638 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ca43f-2cd0-4524-a2b2-bc2f8b33c701 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:26.725983 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7ca43f-2cd0-4524-a2b2-bc2f8b33c701 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:26.727415 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7ca43f-2cd0-4524-a2b2-bc2f8b33c701 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:26.737389 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1343/2669] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:15:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:15:27.163466 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1d21a-a9dc-4f68-ae10-2858dd4cf5cf None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:27.165164 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-32f1d21a-a9dc-4f68-ae10-2858dd4cf5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:27.181350 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1d21a-a9dc-4f68-ae10-2858dd4cf5cf None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:27.195436 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401eecf-7ae2-4c9a-9de4-accd19d013a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:27.196139 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401eecf-7ae2-4c9a-9de4-accd19d013a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:27.196555 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401eecf-7ae2-4c9a-9de4-accd19d013a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:27.196711 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401eecf-7ae2-4c9a-9de4-accd19d013a4 admin admin] PATH_INFO: `/v3/projects/facf2910c1fe4061bb38ecd59c9a70c6` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:27.197911 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b401eecf-7ae2-4c9a-9de4-accd19d013a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=facf2910c1fe4061bb38ecd59c9a70c6)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:27.199980 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b401eecf-7ae2-4c9a-9de4-accd19d013a4 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:27.224398 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:27.333200 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b401eecf-7ae2-4c9a-9de4-accd19d013a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'facf2910c1fe4061bb38ecd59c9a70c6', 'request_id': 'req-b401eecf-7ae2-4c9a-9de4-accd19d013a4'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:15:27.407027 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b401eecf-7ae2-4c9a-9de4-accd19d013a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'facf2910c1fe4061bb38ecd59c9a70c6', 'request_id': 'req-b401eecf-7ae2-4c9a-9de4-accd19d013a4'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:15:27.418951 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b401eecf-7ae2-4c9a-9de4-accd19d013a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'facf2910c1fe4061bb38ecd59c9a70c6', 'request_id': 'req-b401eecf-7ae2-4c9a-9de4-accd19d013a4'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:15:27.432757 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b401eecf-7ae2-4c9a-9de4-accd19d013a4 admin admin] The token cache is being invalidate because project facf2910c1fe4061bb38ecd59c9a70c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:15:27.433509 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b401eecf-7ae2-4c9a-9de4-accd19d013a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:15:27.434184 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b401eecf-7ae2-4c9a-9de4-accd19d013a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:15:27.435791 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1327/2670] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:15:27 2022] DELETE /identity/v3/projects/facf2910c1fe4061bb38ecd59c9a70c6 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:15:27.440877 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f18d87-b8bf-4317-83c6-dd91f290d4af None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:27.443404 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f18d87-b8bf-4317-83c6-dd91f290d4af None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:27.443404 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f18d87-b8bf-4317-83c6-dd91f290d4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:27.462282 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-49f18d87-b8bf-4317-83c6-dd91f290d4af None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:15:27.463519 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-49f18d87-b8bf-4317-83c6-dd91f290d4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:27.559095 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1344/2671] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:15:27.566663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd51f6-d861-4a63-aac8-3625af6e45bf None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:27.567240 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd51f6-d861-4a63-aac8-3625af6e45bf None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:27.567413 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd51f6-d861-4a63-aac8-3625af6e45bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:27.599259 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-5afd51f6-d861-4a63-aac8-3625af6e45bf None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:15:27.601214 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5afd51f6-d861-4a63-aac8-3625af6e45bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:27.610610 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1328/2672] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:15:27.617435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f18d87-b8bf-4317-83c6-dd91f290d4af None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:27.628188 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f18d87-b8bf-4317-83c6-dd91f290d4af None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:27.633666 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee792dbf-bb7e-44d6-835c-29a1b8262345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:27.634397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee792dbf-bb7e-44d6-835c-29a1b8262345 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:27.634777 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee792dbf-bb7e-44d6-835c-29a1b8262345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:27.635145 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee792dbf-bb7e-44d6-835c-29a1b8262345 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:27.635899 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee792dbf-bb7e-44d6-835c-29a1b8262345 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:27.636337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee792dbf-bb7e-44d6-835c-29a1b8262345 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:27.638648 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee792dbf-bb7e-44d6-835c-29a1b8262345 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:27.664191 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1345/2673] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:27.668310 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd51f6-d861-4a63-aac8-3625af6e45bf None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:27.680415 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd51f6-d861-4a63-aac8-3625af6e45bf None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:27.687713 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16265e-e984-42c3-8689-1febe81825f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:27.688306 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16265e-e984-42c3-8689-1febe81825f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:27.688585 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16265e-e984-42c3-8689-1febe81825f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:27.688857 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16265e-e984-42c3-8689-1febe81825f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:27.689506 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df16265e-e984-42c3-8689-1febe81825f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:27.691620 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df16265e-e984-42c3-8689-1febe81825f7 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:27.812723 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1329/2674] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:15:27 2022] POST /identity/v3/projects => generated 371 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:15:27.813958 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee792dbf-bb7e-44d6-835c-29a1b8262345 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:27.825032 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee792dbf-bb7e-44d6-835c-29a1b8262345 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:27.864424 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:27.894826 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54feff-1c3c-4796-bcd0-dafc2e237b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:27.895587 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54feff-1c3c-4796-bcd0-dafc2e237b5b admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:27.895962 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54feff-1c3c-4796-bcd0-dafc2e237b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:27.896538 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54feff-1c3c-4796-bcd0-dafc2e237b5b admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:27.897575 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b54feff-1c3c-4796-bcd0-dafc2e237b5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:27.899890 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b54feff-1c3c-4796-bcd0-dafc2e237b5b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:28.019540 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1346/2675] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:15:27 2022] POST /identity/v3/users => generated 286 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:15:28.023776 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16265e-e984-42c3-8689-1febe81825f7 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:28.035005 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16265e-e984-42c3-8689-1febe81825f7 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:28.039604 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb831a57-de94-447b-9934-d2c216a352e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:28.040272 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb831a57-de94-447b-9934-d2c216a352e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:28.040553 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb831a57-de94-447b-9934-d2c216a352e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:28.040830 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb831a57-de94-447b-9934-d2c216a352e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:28.041483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb831a57-de94-447b-9934-d2c216a352e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:28.043709 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb831a57-de94-447b-9934-d2c216a352e6 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:28.054465 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1330/2676] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:15:28.059111 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54feff-1c3c-4796-bcd0-dafc2e237b5b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:28.070089 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54feff-1c3c-4796-bcd0-dafc2e237b5b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:28.075321 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af2bb6b-fd06-47c3-99c4-6ea4144943f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:28.076020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2bb6b-fd06-47c3-99c4-6ea4144943f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:28.076445 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2bb6b-fd06-47c3-99c4-6ea4144943f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:28.076795 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2bb6b-fd06-47c3-99c4-6ea4144943f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:28.077603 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af2bb6b-fd06-47c3-99c4-6ea4144943f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:28.079574 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af2bb6b-fd06-47c3-99c4-6ea4144943f1 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:28.094632 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1347/2677] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:15:28.095823 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb831a57-de94-447b-9934-d2c216a352e6 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:28.106582 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb831a57-de94-447b-9934-d2c216a352e6 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:28.115942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc4d8c-0a7b-4d35-9ae2-f0628b285cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:28.116760 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc4d8c-0a7b-4d35-9ae2-f0628b285cfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:28.117051 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc4d8c-0a7b-4d35-9ae2-f0628b285cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:28.117362 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc4d8c-0a7b-4d35-9ae2-f0628b285cfe admin admin] PATH_INFO: `/v3/projects/684ee48934f949499a7c77d782372b53/users/3ce5a94e8c594b8d813ab8e6d39633af/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:28.132956 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc4d8c-0a7b-4d35-9ae2-f0628b285cfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=684ee48934f949499a7c77d782372b53, user_id=3ce5a94e8c594b8d813ab8e6d39633af, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:28.135624 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc4d8c-0a7b-4d35-9ae2-f0628b285cfe admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:28.248617 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1331/2678] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:15:28 2022] PUT /identity/v3/projects/684ee48934f949499a7c77d782372b53/users/3ce5a94e8c594b8d813ab8e6d39633af/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:15:28.253884 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff320656-152b-461c-a113-de16948b7ef4 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:28.254197 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff320656-152b-461c-a113-de16948b7ef4 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:28.254470 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff320656-152b-461c-a113-de16948b7ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:28.275344 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ff320656-152b-461c-a113-de16948b7ef4 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:15:28.276677 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ff320656-152b-461c-a113-de16948b7ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:28.340907 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1348/2679] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:15:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:15:28.351133 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc4d8c-0a7b-4d35-9ae2-f0628b285cfe admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:28.352926 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-1fdc4d8c-0a7b-4d35-9ae2-f0628b285cfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:28.415181 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc4d8c-0a7b-4d35-9ae2-f0628b285cfe admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:28.418921 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a35a06-7d34-4fb0-a0d9-cfb71a621dfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:28.419534 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a35a06-7d34-4fb0-a0d9-cfb71a621dfe service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:28.419812 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a35a06-7d34-4fb0-a0d9-cfb71a621dfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:28.420107 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a35a06-7d34-4fb0-a0d9-cfb71a621dfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:28.429861 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a35a06-7d34-4fb0-a0d9-cfb71a621dfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:28.431457 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a35a06-7d34-4fb0-a0d9-cfb71a621dfe service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:28.442926 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1332/2680] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:15:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:15:30.825631 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bfe28-0e0b-4f1e-8b55-e867aaab1ed4 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:30.826207 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bfe28-0e0b-4f1e-8b55-e867aaab1ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:30.826549 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bfe28-0e0b-4f1e-8b55-e867aaab1ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:30.856352 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-872bfe28-0e0b-4f1e-8b55-e867aaab1ed4 None None] MFA Rules not processed for user `3ce5a94e8c594b8d813ab8e6d39633af`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:15:30.857662 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-872bfe28-0e0b-4f1e-8b55-e867aaab1ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:30.925566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1349/2681] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:15:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:15:30.936138 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a35a06-7d34-4fb0-a0d9-cfb71a621dfe service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:30.937526 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-02a35a06-7d34-4fb0-a0d9-cfb71a621dfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:30.977086 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:31.001809 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a35a06-7d34-4fb0-a0d9-cfb71a621dfe service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:31.005410 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29dfb00-2fcd-4138-8c27-54a2c4867e15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:31.006020 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29dfb00-2fcd-4138-8c27-54a2c4867e15 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:31.006364 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29dfb00-2fcd-4138-8c27-54a2c4867e15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:31.006662 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29dfb00-2fcd-4138-8c27-54a2c4867e15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:31.016038 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29dfb00-2fcd-4138-8c27-54a2c4867e15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:31.017516 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29dfb00-2fcd-4138-8c27-54a2c4867e15 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:31.027328 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1333/2682] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:15:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:15:33.885981 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695e1d7-1d1f-407c-9d79-b80b0e39d034 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:33.887014 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695e1d7-1d1f-407c-9d79-b80b0e39d034 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:33.887129 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695e1d7-1d1f-407c-9d79-b80b0e39d034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:33.906442 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-6695e1d7-1d1f-407c-9d79-b80b0e39d034 None None] MFA Rules not processed for user `3ce5a94e8c594b8d813ab8e6d39633af`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:15:33.907646 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6695e1d7-1d1f-407c-9d79-b80b0e39d034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:33.914434 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1350/2683] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:15:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:15:34.754499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29dfb00-2fcd-4138-8c27-54a2c4867e15 service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:34.756013 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f29dfb00-2fcd-4138-8c27-54a2c4867e15 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:34.834944 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29dfb00-2fcd-4138-8c27-54a2c4867e15 service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:34.841057 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148dc86-fd63-4801-ba8c-8842e410a8d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:34.841684 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148dc86-fd63-4801-ba8c-8842e410a8d6 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:34.841959 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148dc86-fd63-4801-ba8c-8842e410a8d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:34.842229 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148dc86-fd63-4801-ba8c-8842e410a8d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:34.844002 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-6148dc86-fd63-4801-ba8c-8842e410a8d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:34.928392 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148dc86-fd63-4801-ba8c-8842e410a8d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:34.929973 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148dc86-fd63-4801-ba8c-8842e410a8d6 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:34.948947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1334/2684] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:15:34 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:15:40.189025 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5decbb28-3d61-4cbb-994a-e15ed9a6c9b4 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:40.189556 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5decbb28-3d61-4cbb-994a-e15ed9a6c9b4 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:40.189881 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5decbb28-3d61-4cbb-994a-e15ed9a6c9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:40.247394 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5decbb28-3d61-4cbb-994a-e15ed9a6c9b4 None None] MFA Rules not processed for user `8c21ea7d3cbb494886e60a4bd4f39145`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:15:40.248742 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5decbb28-3d61-4cbb-994a-e15ed9a6c9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:40.254418 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:40.317926 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1351/2685] 10.222.0.48 () {58 vars in 967 bytes} [Fri Sep 9 21:15:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:15:40.330708 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148dc86-fd63-4801-ba8c-8842e410a8d6 service nova] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:40.332256 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-6148dc86-fd63-4801-ba8c-8842e410a8d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:40.385141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:40.422821 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148dc86-fd63-4801-ba8c-8842e410a8d6 service nova] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:40.428706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8a397-a615-47a6-bb67-d6383edc337d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d28fa5d49d44120bd5eaec2d4e5fdbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:40.429346 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8a397-a615-47a6-bb67-d6383edc337d service swift] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:40.429626 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8a397-a615-47a6-bb67-d6383edc337d service swift] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:40.429914 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8a397-a615-47a6-bb67-d6383edc337d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:40.438578 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8a397-a615-47a6-bb67-d6383edc337d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:40.440267 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8a397-a615-47a6-bb67-d6383edc337d service swift] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:40.448240 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1335/2686] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Sep 9 21:15:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 21:15:50.383482 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=24 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:50.445030 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=20 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:59.375225 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5545997-46c6-40d6-9566-3ec575f794e3 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:59.375679 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5545997-46c6-40d6-9566-3ec575f794e3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:59.375774 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5545997-46c6-40d6-9566-3ec575f794e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:59.394077 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c5545997-46c6-40d6-9566-3ec575f794e3 None None] MFA Rules not processed for user `3ce5a94e8c594b8d813ab8e6d39633af`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:15:59.395261 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c5545997-46c6-40d6-9566-3ec575f794e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:59.402024 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1352/2687] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:15:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:16:01.600773 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1510f-ae4b-4fbf-8ba2-3717ab5528ad None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:01.601394 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1510f-ae4b-4fbf-8ba2-3717ab5528ad None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:01.601394 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1510f-ae4b-4fbf-8ba2-3717ab5528ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:01.620579 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-89e1510f-ae4b-4fbf-8ba2-3717ab5528ad None None] MFA Rules not processed for user `f289a1bdd79b4ba0bf9499edb7729c9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:16:01.621711 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-89e1510f-ae4b-4fbf-8ba2-3717ab5528ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:16:01.675974 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1336/2688] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:16:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:16:01.991674 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5545997-46c6-40d6-9566-3ec575f794e3 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:16:01.993179 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c5545997-46c6-40d6-9566-3ec575f794e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:16:02.008483 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5545997-46c6-40d6-9566-3ec575f794e3 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:16:02.009382 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e5d36-349e-4684-8713-a7196eb391e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:16:02.010007 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e5d36-349e-4684-8713-a7196eb391e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:02.010284 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e5d36-349e-4684-8713-a7196eb391e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:02.010554 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e5d36-349e-4684-8713-a7196eb391e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:02.012489 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-332e5d36-349e-4684-8713-a7196eb391e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:16:02.022703 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e5d36-349e-4684-8713-a7196eb391e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:16:02.024231 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e5d36-349e-4684-8713-a7196eb391e3 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:16:02.032692 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1353/2689] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:16:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:16:03.266699 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e1510f-ae4b-4fbf-8ba2-3717ab5528ad None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:16:03.276328 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e1510f-ae4b-4fbf-8ba2-3717ab5528ad None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:16:03.279830 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d784b-6039-47ee-b569-f9ec5350103e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:16:03.279830 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d784b-6039-47ee-b569-f9ec5350103e service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:03.280820 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d784b-6039-47ee-b569-f9ec5350103e service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:03.280977 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d784b-6039-47ee-b569-f9ec5350103e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:03.283302 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a23d784b-6039-47ee-b569-f9ec5350103e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:16:03.297860 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23d784b-6039-47ee-b569-f9ec5350103e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:16:03.297860 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23d784b-6039-47ee-b569-f9ec5350103e service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:16:03.304430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1337/2690] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:16:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:16:12.028654 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=9 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:16:13.300613 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=31 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:16:30.063843 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e5d36-349e-4684-8713-a7196eb391e3 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:16:30.063843 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e5d36-349e-4684-8713-a7196eb391e3 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:16:30.063843 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10106a87-51a5-4c78-bdc4-c63f29b9a60d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:16:30.063843 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10106a87-51a5-4c78-bdc4-c63f29b9a60d service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:30.064908 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10106a87-51a5-4c78-bdc4-c63f29b9a60d service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:30.064908 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10106a87-51a5-4c78-bdc4-c63f29b9a60d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:30.064908 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-10106a87-51a5-4c78-bdc4-c63f29b9a60d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:16:30.064908 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10106a87-51a5-4c78-bdc4-c63f29b9a60d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:16:30.064908 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10106a87-51a5-4c78-bdc4-c63f29b9a60d service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:16:30.064908 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1354/2691] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:16:29 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:16:30.785703 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66616a93-e298-4a7c-95ad-f7f96cdddab2 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:30.786236 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66616a93-e298-4a7c-95ad-f7f96cdddab2 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:30.786514 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66616a93-e298-4a7c-95ad-f7f96cdddab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:30.813727 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-66616a93-e298-4a7c-95ad-f7f96cdddab2 None None] MFA Rules not processed for user `8c21ea7d3cbb494886e60a4bd4f39145`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:16:30.814989 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-66616a93-e298-4a7c-95ad-f7f96cdddab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:16:30.822407 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1338/2692] 10.222.0.48 () {58 vars in 1009 bytes} [Fri Sep 9 21:16:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:16:30.847167 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10106a87-51a5-4c78-bdc4-c63f29b9a60d service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:16:30.858846 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10106a87-51a5-4c78-bdc4-c63f29b9a60d service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:16:30.864036 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcdddd2-f859-4708-863e-02d8e685d78f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d28fa5d49d44120bd5eaec2d4e5fdbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:16:30.864690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcdddd2-f859-4708-863e-02d8e685d78f service swift] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:30.865020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcdddd2-f859-4708-863e-02d8e685d78f service swift] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:30.865309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcdddd2-f859-4708-863e-02d8e685d78f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:30.873631 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcdddd2-f859-4708-863e-02d8e685d78f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:16:30.876105 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcdddd2-f859-4708-863e-02d8e685d78f service swift] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:16:30.885424 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1355/2693] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Sep 9 21:16:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 21:16:33.574995 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66616a93-e298-4a7c-95ad-f7f96cdddab2 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:16:33.590031 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66616a93-e298-4a7c-95ad-f7f96cdddab2 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:16:33.598935 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7137884-79de-44be-b880-3a08b58bbaae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:16:33.600096 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7137884-79de-44be-b880-3a08b58bbaae service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:33.600400 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7137884-79de-44be-b880-3a08b58bbaae service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:33.600679 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7137884-79de-44be-b880-3a08b58bbaae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:33.603355 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a7137884-79de-44be-b880-3a08b58bbaae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:16:33.622467 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7137884-79de-44be-b880-3a08b58bbaae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:16:33.624223 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7137884-79de-44be-b880-3a08b58bbaae service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:16:33.665120 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1339/2694] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:16:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:16:37.847860 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcdddd2-f859-4708-863e-02d8e685d78f service swift] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:16:37.855220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7bcdddd2-f859-4708-863e-02d8e685d78f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:16:37.871095 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcdddd2-f859-4708-863e-02d8e685d78f service swift] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:16:37.876632 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8f41bf-9166-413e-b5e7-387ee8b49b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:16:37.877816 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f41bf-9166-413e-b5e7-387ee8b49b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:37.878184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f41bf-9166-413e-b5e7-387ee8b49b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:37.878518 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f41bf-9166-413e-b5e7-387ee8b49b99 admin admin] PATH_INFO: `/v3/users/e3ef96f764244e5893f16a74d4169edb` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:37.898909 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8f41bf-9166-413e-b5e7-387ee8b49b99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3ef96f764244e5893f16a74d4169edb)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:16:37.900948 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8f41bf-9166-413e-b5e7-387ee8b49b99 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:16:38.339151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4d8f41bf-9166-413e-b5e7-387ee8b49b99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3ef96f764244e5893f16a74d4169edb'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:16:38.339870 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4d8f41bf-9166-413e-b5e7-387ee8b49b99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3ef96f764244e5893f16a74d4169edb'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:16:38.340498 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4d8f41bf-9166-413e-b5e7-387ee8b49b99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3ef96f764244e5893f16a74d4169edb'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:16:38.351739 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4d8f41bf-9166-413e-b5e7-387ee8b49b99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3ef96f764244e5893f16a74d4169edb'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:16:38.368579 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4d8f41bf-9166-413e-b5e7-387ee8b49b99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3ef96f764244e5893f16a74d4169edb'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:16:38.447261 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1356/2695] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:16:37 2022] DELETE /identity/v3/users/e3ef96f764244e5893f16a74d4169edb => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:16:38.780795 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7137884-79de-44be-b880-3a08b58bbaae service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:16:38.795205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7137884-79de-44be-b880-3a08b58bbaae service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:16:38.901150 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624b2cd-dc02-4a39-81f7-b49953b74047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:16:38.904494 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624b2cd-dc02-4a39-81f7-b49953b74047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:38.904821 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624b2cd-dc02-4a39-81f7-b49953b74047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:38.905112 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624b2cd-dc02-4a39-81f7-b49953b74047 admin admin] PATH_INFO: `/v3/projects/d9f70ceb84fc40fbb499b6a80bc5c13b` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:38.917928 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624b2cd-dc02-4a39-81f7-b49953b74047 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9f70ceb84fc40fbb499b6a80bc5c13b)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:16:38.919867 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624b2cd-dc02-4a39-81f7-b49953b74047 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:16:39.026608 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2624b2cd-dc02-4a39-81f7-b49953b74047 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9f70ceb84fc40fbb499b6a80bc5c13b', 'request_id': 'req-2624b2cd-dc02-4a39-81f7-b49953b74047'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:16:39.109265 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2624b2cd-dc02-4a39-81f7-b49953b74047 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9f70ceb84fc40fbb499b6a80bc5c13b', 'request_id': 'req-2624b2cd-dc02-4a39-81f7-b49953b74047'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:16:39.117145 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2624b2cd-dc02-4a39-81f7-b49953b74047 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9f70ceb84fc40fbb499b6a80bc5c13b', 'request_id': 'req-2624b2cd-dc02-4a39-81f7-b49953b74047'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:16:39.129336 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2624b2cd-dc02-4a39-81f7-b49953b74047 admin admin] The token cache is being invalidate because project d9f70ceb84fc40fbb499b6a80bc5c13b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:16:39.129638 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2624b2cd-dc02-4a39-81f7-b49953b74047 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:16:39.130304 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2624b2cd-dc02-4a39-81f7-b49953b74047 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:16:39.131802 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1340/2696] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:16:38 2022] DELETE /identity/v3/projects/d9f70ceb84fc40fbb499b6a80bc5c13b => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:16:39.138725 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894aaa8-a629-4214-8cac-fed13e016824 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:39.138725 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894aaa8-a629-4214-8cac-fed13e016824 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:39.138725 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894aaa8-a629-4214-8cac-fed13e016824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:39.159965 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b894aaa8-a629-4214-8cac-fed13e016824 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:16:39.161296 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b894aaa8-a629-4214-8cac-fed13e016824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:16:39.209189 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:16:39.236335 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1357/2697] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:16:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:16:39.247302 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a18db3-a1b8-4a3b-90f1-249df05cae7d None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:39.247592 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a18db3-a1b8-4a3b-90f1-249df05cae7d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:39.247896 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a18db3-a1b8-4a3b-90f1-249df05cae7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:39.268257 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-45a18db3-a1b8-4a3b-90f1-249df05cae7d None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:16:39.269504 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-45a18db3-a1b8-4a3b-90f1-249df05cae7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:16:39.280767 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1341/2698] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:16:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:16:39.282037 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894aaa8-a629-4214-8cac-fed13e016824 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:16:39.292179 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894aaa8-a629-4214-8cac-fed13e016824 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:16:39.297882 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ec7ec-d793-4560-8a17-ee8e0af223ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:16:39.298490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ec7ec-d793-4560-8a17-ee8e0af223ea admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:39.298764 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ec7ec-d793-4560-8a17-ee8e0af223ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:39.299031 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ec7ec-d793-4560-8a17-ee8e0af223ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:39.299851 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ec7ec-d793-4560-8a17-ee8e0af223ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:16:39.300227 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ec7ec-d793-4560-8a17-ee8e0af223ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:16:39.301815 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ec7ec-d793-4560-8a17-ee8e0af223ea admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:16:39.316891 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1358/2699] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:16:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:16:39.321089 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a18db3-a1b8-4a3b-90f1-249df05cae7d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:16:39.333766 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a18db3-a1b8-4a3b-90f1-249df05cae7d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:16:39.340047 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d597e9-a8f9-4117-9304-d54a01cffd4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:16:39.340871 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d597e9-a8f9-4117-9304-d54a01cffd4c admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:39.341401 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d597e9-a8f9-4117-9304-d54a01cffd4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:39.341846 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d597e9-a8f9-4117-9304-d54a01cffd4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:39.342730 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d597e9-a8f9-4117-9304-d54a01cffd4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:16:39.345342 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d597e9-a8f9-4117-9304-d54a01cffd4c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:16:39.438710 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1342/2700] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:16:39 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:16:39.443638 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ec7ec-d793-4560-8a17-ee8e0af223ea admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:16:39.457623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ec7ec-d793-4560-8a17-ee8e0af223ea admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:16:39.526948 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08494fe-45d4-4bb2-88f7-37c4055afc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:16:39.527696 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08494fe-45d4-4bb2-88f7-37c4055afc8b admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:39.528000 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08494fe-45d4-4bb2-88f7-37c4055afc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:39.528307 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08494fe-45d4-4bb2-88f7-37c4055afc8b admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:39.528951 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08494fe-45d4-4bb2-88f7-37c4055afc8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:16:39.531181 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08494fe-45d4-4bb2-88f7-37c4055afc8b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:16:39.621560 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1359/2701] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:16:39 2022] POST /identity/v3/users => generated 287 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:16:39.624148 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d597e9-a8f9-4117-9304-d54a01cffd4c admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:16:39.634457 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d597e9-a8f9-4117-9304-d54a01cffd4c admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:16:39.639291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f67615-551e-44fc-92ad-81dfea9d5bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:16:39.639899 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f67615-551e-44fc-92ad-81dfea9d5bd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:39.640865 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f67615-551e-44fc-92ad-81dfea9d5bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:39.641265 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f67615-551e-44fc-92ad-81dfea9d5bd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:39.641912 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f67615-551e-44fc-92ad-81dfea9d5bd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:16:39.643495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f67615-551e-44fc-92ad-81dfea9d5bd2 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:16:39.656126 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1343/2702] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:16:39.658904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08494fe-45d4-4bb2-88f7-37c4055afc8b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:16:39.669476 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08494fe-45d4-4bb2-88f7-37c4055afc8b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:16:39.674536 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b694d-1444-4332-affa-e141010ccdf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:16:39.675252 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b694d-1444-4332-affa-e141010ccdf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:39.675635 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b694d-1444-4332-affa-e141010ccdf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:39.676007 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b694d-1444-4332-affa-e141010ccdf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:39.676796 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b694d-1444-4332-affa-e141010ccdf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:16:39.678487 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b694d-1444-4332-affa-e141010ccdf2 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:16:39.692548 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1360/2703] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:16:39.697365 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f67615-551e-44fc-92ad-81dfea9d5bd2 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:16:39.708877 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f67615-551e-44fc-92ad-81dfea9d5bd2 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:16:39.714007 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4268c-aad9-486c-92b7-221dfc2e4863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:16:39.714787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4268c-aad9-486c-92b7-221dfc2e4863 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:39.715177 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4268c-aad9-486c-92b7-221dfc2e4863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:39.715536 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4268c-aad9-486c-92b7-221dfc2e4863 admin admin] PATH_INFO: `/v3/projects/536bf5a46b4d4ca69ddf05a2f617ee9e/users/35ee93a615ff4485a500f99d7fbb273d/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:39.735352 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a4268c-aad9-486c-92b7-221dfc2e4863 admin admin] RBAC: Authorizing `identity:create_grant(project_id=536bf5a46b4d4ca69ddf05a2f617ee9e, user_id=35ee93a615ff4485a500f99d7fbb273d, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:16:39.738185 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a4268c-aad9-486c-92b7-221dfc2e4863 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:16:39.799326 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:16:39.816577 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1344/2704] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:16:39 2022] PUT /identity/v3/projects/536bf5a46b4d4ca69ddf05a2f617ee9e/users/35ee93a615ff4485a500f99d7fbb273d/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:16:39.818294 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebcb953-4c43-4e88-99bf-c4b80e7a090d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:39.818702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebcb953-4c43-4e88-99bf-c4b80e7a090d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:39.819009 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebcb953-4c43-4e88-99bf-c4b80e7a090d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:39.855990 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5ebcb953-4c43-4e88-99bf-c4b80e7a090d None None] MFA Rules not processed for user `35ee93a615ff4485a500f99d7fbb273d`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:16:39.857763 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5ebcb953-4c43-4e88-99bf-c4b80e7a090d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:16:39.941440 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1361/2705] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:16:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:16:39.946599 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4268c-aad9-486c-92b7-221dfc2e4863 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:16:40.027509 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4268c-aad9-486c-92b7-221dfc2e4863 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:16:40.032556 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f74429-6c75-4f9e-a4d6-9bf0ce16052b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:16:40.033226 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f74429-6c75-4f9e-a4d6-9bf0ce16052b admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:40.033633 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f74429-6c75-4f9e-a4d6-9bf0ce16052b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:40.033933 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f74429-6c75-4f9e-a4d6-9bf0ce16052b admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:40.034581 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f74429-6c75-4f9e-a4d6-9bf0ce16052b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:16:40.036584 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f74429-6c75-4f9e-a4d6-9bf0ce16052b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:16:40.135358 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1345/2706] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:16:39 2022] POST /identity/v3/projects => generated 373 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:16:40.139260 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebcb953-4c43-4e88-99bf-c4b80e7a090d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:16:40.152677 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebcb953-4c43-4e88-99bf-c4b80e7a090d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:16:40.192432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:16:40.218557 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec11f8-8660-4e50-86ef-b607f43c95c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:16:40.219163 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec11f8-8660-4e50-86ef-b607f43c95c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:40.219434 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec11f8-8660-4e50-86ef-b607f43c95c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:40.219672 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec11f8-8660-4e50-86ef-b607f43c95c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:40.220310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ec11f8-8660-4e50-86ef-b607f43c95c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:16:40.222518 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ec11f8-8660-4e50-86ef-b607f43c95c0 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:16:40.317606 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1362/2707] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:16:40 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:16:40.321798 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f74429-6c75-4f9e-a4d6-9bf0ce16052b admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:16:40.337296 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f74429-6c75-4f9e-a4d6-9bf0ce16052b admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:16:40.346889 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c8d82-4119-460d-8460-235f5b8650c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:16:40.347883 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c8d82-4119-460d-8460-235f5b8650c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:40.348268 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c8d82-4119-460d-8460-235f5b8650c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:40.348628 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c8d82-4119-460d-8460-235f5b8650c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:40.349372 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c8d82-4119-460d-8460-235f5b8650c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:16:40.351041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c8d82-4119-460d-8460-235f5b8650c0 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:16:40.366587 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1346/2708] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:16:40.367914 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec11f8-8660-4e50-86ef-b607f43c95c0 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:16:40.379148 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec11f8-8660-4e50-86ef-b607f43c95c0 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:16:40.389337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd923e0-bfbc-4ca2-953f-c3cc98bff2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:16:40.390025 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd923e0-bfbc-4ca2-953f-c3cc98bff2f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:40.390934 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd923e0-bfbc-4ca2-953f-c3cc98bff2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:40.391267 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd923e0-bfbc-4ca2-953f-c3cc98bff2f7 admin admin] PATH_INFO: `/v3/projects/fb1d56b893cf45729d68dcb5fef86dc5/users/6aaea8d20e2241f88ebba26a853b9d76/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:40.409731 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd923e0-bfbc-4ca2-953f-c3cc98bff2f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb1d56b893cf45729d68dcb5fef86dc5, user_id=6aaea8d20e2241f88ebba26a853b9d76, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:16:40.412437 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd923e0-bfbc-4ca2-953f-c3cc98bff2f7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:16:40.487261 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1363/2709] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:16:40 2022] PUT /identity/v3/projects/fb1d56b893cf45729d68dcb5fef86dc5/users/6aaea8d20e2241f88ebba26a853b9d76/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:16:40.491536 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c8d82-4119-460d-8460-235f5b8650c0 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:16:40.504487 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c8d82-4119-460d-8460-235f5b8650c0 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:16:40.569849 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb785b-a556-44ea-91b6-0fbae16057bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:16:40.570808 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb785b-a556-44ea-91b6-0fbae16057bb admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:40.571376 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb785b-a556-44ea-91b6-0fbae16057bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:40.571832 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb785b-a556-44ea-91b6-0fbae16057bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:40.572711 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcb785b-a556-44ea-91b6-0fbae16057bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:16:40.574561 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcb785b-a556-44ea-91b6-0fbae16057bb admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:16:40.586614 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1347/2710] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:16:40.591810 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd923e0-bfbc-4ca2-953f-c3cc98bff2f7 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:16:40.602331 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd923e0-bfbc-4ca2-953f-c3cc98bff2f7 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:16:40.607481 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c9844-dd2d-498b-ab0c-8917163318ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:16:40.608294 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c9844-dd2d-498b-ab0c-8917163318ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:40.608696 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c9844-dd2d-498b-ab0c-8917163318ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:40.609091 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c9844-dd2d-498b-ab0c-8917163318ca admin admin] PATH_INFO: `/v3/projects/fb1d56b893cf45729d68dcb5fef86dc5/users/6aaea8d20e2241f88ebba26a853b9d76/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:40.611207 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c9844-dd2d-498b-ab0c-8917163318ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb1d56b893cf45729d68dcb5fef86dc5, user_id=6aaea8d20e2241f88ebba26a853b9d76, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:16:40.614059 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c9844-dd2d-498b-ab0c-8917163318ca admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:16:40.627706 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1364/2711] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:16:40 2022] PUT /identity/v3/projects/fb1d56b893cf45729d68dcb5fef86dc5/users/6aaea8d20e2241f88ebba26a853b9d76/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:16:40.637900 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ef16b-15bd-483c-9d62-02865232516c None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:40.638188 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ef16b-15bd-483c-9d62-02865232516c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:40.638465 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ef16b-15bd-483c-9d62-02865232516c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:40.670158 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-077ef16b-15bd-483c-9d62-02865232516c None None] MFA Rules not processed for user `6aaea8d20e2241f88ebba26a853b9d76`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:16:40.671965 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-077ef16b-15bd-483c-9d62-02865232516c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:16:40.746070 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:16:40.764161 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1348/2712] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:16:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:16:40.764829 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c9844-dd2d-498b-ab0c-8917163318ca admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:16:40.766801 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b17c9844-dd2d-498b-ab0c-8917163318ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:16:40.841711 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c9844-dd2d-498b-ab0c-8917163318ca admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:16:40.847928 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d35bf7-3dfd-4f1a-be15-45c790ec34dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:16:40.854848 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d35bf7-3dfd-4f1a-be15-45c790ec34dc service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:40.854848 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d35bf7-3dfd-4f1a-be15-45c790ec34dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:40.854848 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d35bf7-3dfd-4f1a-be15-45c790ec34dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:40.866631 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d35bf7-3dfd-4f1a-be15-45c790ec34dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:16:40.868443 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d35bf7-3dfd-4f1a-be15-45c790ec34dc service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:16:40.882801 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1365/2713] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:16:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:16:41.290049 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ef16b-15bd-483c-9d62-02865232516c None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:16:41.299828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ef16b-15bd-483c-9d62-02865232516c None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:16:41.303441 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27e50c-ad96-4a61-8755-c880bc350483 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:16:41.304047 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27e50c-ad96-4a61-8755-c880bc350483 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:41.304343 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27e50c-ad96-4a61-8755-c880bc350483 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:41.304612 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27e50c-ad96-4a61-8755-c880bc350483 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:41.381148 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf27e50c-ad96-4a61-8755-c880bc350483 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:16:41.382582 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf27e50c-ad96-4a61-8755-c880bc350483 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:16:41.397412 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1349/2714] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:16:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:16:50.244235 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03201e-88ce-4916-8779-f7da7e2c3784 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:50.244710 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03201e-88ce-4916-8779-f7da7e2c3784 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:50.244823 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03201e-88ce-4916-8779-f7da7e2c3784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:50.266518 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-be03201e-88ce-4916-8779-f7da7e2c3784 None None] MFA Rules not processed for user `8c21ea7d3cbb494886e60a4bd4f39145`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:16:50.267709 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-be03201e-88ce-4916-8779-f7da7e2c3784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:16:50.322195 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1366/2715] 10.222.0.48 () {58 vars in 967 bytes} [Fri Sep 9 21:16:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:16:50.334386 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27e50c-ad96-4a61-8755-c880bc350483 service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:16:50.335790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-cf27e50c-ad96-4a61-8755-c880bc350483 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:16:50.402509 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27e50c-ad96-4a61-8755-c880bc350483 service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:16:50.408200 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4c31d-2043-4378-a6fb-3b66ab53bacc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d28fa5d49d44120bd5eaec2d4e5fdbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:16:50.408817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4c31d-2043-4378-a6fb-3b66ab53bacc service swift] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:50.409101 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4c31d-2043-4378-a6fb-3b66ab53bacc service swift] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:50.409386 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4c31d-2043-4378-a6fb-3b66ab53bacc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:50.417890 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb4c31d-2043-4378-a6fb-3b66ab53bacc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:16:50.419470 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb4c31d-2043-4378-a6fb-3b66ab53bacc service swift] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:16:50.440519 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1350/2716] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Sep 9 21:16:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 21:16:53.309025 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03201e-88ce-4916-8779-f7da7e2c3784 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:16:53.310567 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-be03201e-88ce-4916-8779-f7da7e2c3784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:16:53.353797 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:16:53.377369 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03201e-88ce-4916-8779-f7da7e2c3784 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:16:53.381913 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f43c2-1a8c-4f9c-9b49-c5f8e9d35c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:16:53.382553 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f43c2-1a8c-4f9c-9b49-c5f8e9d35c81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:53.382811 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f43c2-1a8c-4f9c-9b49-c5f8e9d35c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:53.383079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f43c2-1a8c-4f9c-9b49-c5f8e9d35c81 admin admin] PATH_INFO: `/v3/users/3ce5a94e8c594b8d813ab8e6d39633af` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:53.384256 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f43c2-1a8c-4f9c-9b49-c5f8e9d35c81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ce5a94e8c594b8d813ab8e6d39633af)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:16:53.386173 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f43c2-1a8c-4f9c-9b49-c5f8e9d35c81 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:16:53.545910 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8a9f43c2-1a8c-4f9c-9b49-c5f8e9d35c81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ce5a94e8c594b8d813ab8e6d39633af'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:16:53.546618 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8a9f43c2-1a8c-4f9c-9b49-c5f8e9d35c81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ce5a94e8c594b8d813ab8e6d39633af'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:16:53.547331 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8a9f43c2-1a8c-4f9c-9b49-c5f8e9d35c81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ce5a94e8c594b8d813ab8e6d39633af'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:16:53.554556 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8a9f43c2-1a8c-4f9c-9b49-c5f8e9d35c81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ce5a94e8c594b8d813ab8e6d39633af'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:16:53.566310 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8a9f43c2-1a8c-4f9c-9b49-c5f8e9d35c81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ce5a94e8c594b8d813ab8e6d39633af'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:16:53.610331 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1367/2717] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:16:53 2022] DELETE /identity/v3/users/3ce5a94e8c594b8d813ab8e6d39633af => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:16:53.857547 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4c31d-2043-4378-a6fb-3b66ab53bacc service swift] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:16:53.868440 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4c31d-2043-4378-a6fb-3b66ab53bacc service swift] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:16:53.932867 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8169d0-a447-4702-979b-0d9042f0a5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:16:53.933511 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8169d0-a447-4702-979b-0d9042f0a5be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:53.933793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8169d0-a447-4702-979b-0d9042f0a5be admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:53.934068 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8169d0-a447-4702-979b-0d9042f0a5be admin admin] PATH_INFO: `/v3/projects/684ee48934f949499a7c77d782372b53` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:53.935201 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8169d0-a447-4702-979b-0d9042f0a5be admin admin] RBAC: Authorizing `identity:delete_project(project_id=684ee48934f949499a7c77d782372b53)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:16:53.937147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8169d0-a447-4702-979b-0d9042f0a5be admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:16:53.961449 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:16:54.064828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8a8169d0-a447-4702-979b-0d9042f0a5be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '684ee48934f949499a7c77d782372b53', 'request_id': 'req-8a8169d0-a447-4702-979b-0d9042f0a5be'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:16:54.143358 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8a8169d0-a447-4702-979b-0d9042f0a5be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '684ee48934f949499a7c77d782372b53', 'request_id': 'req-8a8169d0-a447-4702-979b-0d9042f0a5be'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:16:54.150258 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8a8169d0-a447-4702-979b-0d9042f0a5be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '684ee48934f949499a7c77d782372b53', 'request_id': 'req-8a8169d0-a447-4702-979b-0d9042f0a5be'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:16:54.161290 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8a8169d0-a447-4702-979b-0d9042f0a5be admin admin] The token cache is being invalidate because project 684ee48934f949499a7c77d782372b53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:16:54.161576 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8a8169d0-a447-4702-979b-0d9042f0a5be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:16:54.162195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8a8169d0-a447-4702-979b-0d9042f0a5be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:16:54.163660 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1351/2718] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:16:53 2022] DELETE /identity/v3/projects/684ee48934f949499a7c77d782372b53 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:17:03.606379 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:04.158114 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=15,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:08.504908 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f43c2-1a8c-4f9c-9b49-c5f8e9d35c81 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:08.506336 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8a9f43c2-1a8c-4f9c-9b49-c5f8e9d35c81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:08.563034 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f43c2-1a8c-4f9c-9b49-c5f8e9d35c81 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:08.610887 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb7349-2404-4d20-9d17-a67ed4b84da3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:08.611539 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb7349-2404-4d20-9d17-a67ed4b84da3 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:08.611949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb7349-2404-4d20-9d17-a67ed4b84da3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:08.612268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb7349-2404-4d20-9d17-a67ed4b84da3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:08.614567 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6dbb7349-2404-4d20-9d17-a67ed4b84da3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:08.681897 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb7349-2404-4d20-9d17-a67ed4b84da3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:08.683339 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb7349-2404-4d20-9d17-a67ed4b84da3 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:08.693549 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1368/2719] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:17:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:17:12.218459 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8169d0-a447-4702-979b-0d9042f0a5be admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:12.219876 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8a8169d0-a447-4702-979b-0d9042f0a5be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:12.234652 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8169d0-a447-4702-979b-0d9042f0a5be admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:12.239281 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97678636-5283-4a13-b0c8-8c8e2e7a4d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:12.239917 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97678636-5283-4a13-b0c8-8c8e2e7a4d02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:12.240242 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97678636-5283-4a13-b0c8-8c8e2e7a4d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:12.240520 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97678636-5283-4a13-b0c8-8c8e2e7a4d02 admin admin] PATH_INFO: `/v3/users/710e49db188845bbae5a183c6657e07f` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:12.241655 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97678636-5283-4a13-b0c8-8c8e2e7a4d02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=710e49db188845bbae5a183c6657e07f)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:12.243642 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97678636-5283-4a13-b0c8-8c8e2e7a4d02 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:12.378930 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-97678636-5283-4a13-b0c8-8c8e2e7a4d02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '710e49db188845bbae5a183c6657e07f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:12.380177 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-97678636-5283-4a13-b0c8-8c8e2e7a4d02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '710e49db188845bbae5a183c6657e07f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:12.387193 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-97678636-5283-4a13-b0c8-8c8e2e7a4d02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '710e49db188845bbae5a183c6657e07f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:12.399731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-97678636-5283-4a13-b0c8-8c8e2e7a4d02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '710e49db188845bbae5a183c6657e07f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:12.400377 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-97678636-5283-4a13-b0c8-8c8e2e7a4d02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '710e49db188845bbae5a183c6657e07f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:12.474092 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1352/2720] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:17:12 2022] DELETE /identity/v3/users/710e49db188845bbae5a183c6657e07f => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:17:12.965797 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb7349-2404-4d20-9d17-a67ed4b84da3 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:12.978431 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb7349-2404-4d20-9d17-a67ed4b84da3 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:13.037666 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64aa738-52f7-4b49-9557-a915bdf6f96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:13.038313 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64aa738-52f7-4b49-9557-a915bdf6f96d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:13.038592 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64aa738-52f7-4b49-9557-a915bdf6f96d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:13.038864 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64aa738-52f7-4b49-9557-a915bdf6f96d admin admin] PATH_INFO: `/v3/projects/620d0a2c4566408aa134f2a8ac7d91ee` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:13.039985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64aa738-52f7-4b49-9557-a915bdf6f96d admin admin] RBAC: Authorizing `identity:delete_project(project_id=620d0a2c4566408aa134f2a8ac7d91ee)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:13.041942 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64aa738-52f7-4b49-9557-a915bdf6f96d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:13.049985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:13.177957 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c64aa738-52f7-4b49-9557-a915bdf6f96d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '620d0a2c4566408aa134f2a8ac7d91ee', 'request_id': 'req-c64aa738-52f7-4b49-9557-a915bdf6f96d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:13.184150 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c64aa738-52f7-4b49-9557-a915bdf6f96d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '620d0a2c4566408aa134f2a8ac7d91ee', 'request_id': 'req-c64aa738-52f7-4b49-9557-a915bdf6f96d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:13.255443 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c64aa738-52f7-4b49-9557-a915bdf6f96d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '620d0a2c4566408aa134f2a8ac7d91ee', 'request_id': 'req-c64aa738-52f7-4b49-9557-a915bdf6f96d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:13.265994 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c64aa738-52f7-4b49-9557-a915bdf6f96d admin admin] The token cache is being invalidate because project 620d0a2c4566408aa134f2a8ac7d91ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:17:13.266286 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c64aa738-52f7-4b49-9557-a915bdf6f96d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:13.266916 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c64aa738-52f7-4b49-9557-a915bdf6f96d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:13.268513 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1369/2721] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:17:12 2022] DELETE /identity/v3/projects/620d0a2c4566408aa134f2a8ac7d91ee => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:17:13.275456 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ef34e-71af-4e9d-9056-9883d4f178a7 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:13.275456 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ef34e-71af-4e9d-9056-9883d4f178a7 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:13.275456 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ef34e-71af-4e9d-9056-9883d4f178a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:13.294350 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-362ef34e-71af-4e9d-9056-9883d4f178a7 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:13.295555 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-362ef34e-71af-4e9d-9056-9883d4f178a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:13.353275 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1353/2722] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:17:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:17:13.361679 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3ce26-9007-4bb9-97e9-724854654d46 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:13.361961 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3ce26-9007-4bb9-97e9-724854654d46 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:13.362233 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3ce26-9007-4bb9-97e9-724854654d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:13.381419 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-53a3ce26-9007-4bb9-97e9-724854654d46 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:13.382607 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-53a3ce26-9007-4bb9-97e9-724854654d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:13.390587 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1370/2723] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:17:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:17:13.395284 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ef34e-71af-4e9d-9056-9883d4f178a7 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:13.404840 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ef34e-71af-4e9d-9056-9883d4f178a7 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:13.409412 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81965082-bfb1-4438-af29-b77eccb6d372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:13.409997 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81965082-bfb1-4438-af29-b77eccb6d372 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:13.410284 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81965082-bfb1-4438-af29-b77eccb6d372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:13.411138 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81965082-bfb1-4438-af29-b77eccb6d372 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:13.411777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81965082-bfb1-4438-af29-b77eccb6d372 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:17:13.412102 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81965082-bfb1-4438-af29-b77eccb6d372 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:13.413683 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81965082-bfb1-4438-af29-b77eccb6d372 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:13.424899 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1354/2724] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:17:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:17:13.428868 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3ce26-9007-4bb9-97e9-724854654d46 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:13.438189 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3ce26-9007-4bb9-97e9-724854654d46 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:13.442743 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520e54a-3990-4275-bde0-5db5194639e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:13.443328 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520e54a-3990-4275-bde0-5db5194639e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:13.443601 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520e54a-3990-4275-bde0-5db5194639e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:13.443895 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520e54a-3990-4275-bde0-5db5194639e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:13.444557 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4520e54a-3990-4275-bde0-5db5194639e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:13.446462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4520e54a-3990-4275-bde0-5db5194639e4 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:13.573022 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1371/2725] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:17:13 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:17:13.577255 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81965082-bfb1-4438-af29-b77eccb6d372 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:13.588009 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81965082-bfb1-4438-af29-b77eccb6d372 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:13.644495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543860cd-81ab-480b-ae01-f1ec3823fcf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:13.645656 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543860cd-81ab-480b-ae01-f1ec3823fcf5 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:13.645936 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543860cd-81ab-480b-ae01-f1ec3823fcf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:13.646210 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543860cd-81ab-480b-ae01-f1ec3823fcf5 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:13.646835 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543860cd-81ab-480b-ae01-f1ec3823fcf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:13.648810 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543860cd-81ab-480b-ae01-f1ec3823fcf5 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:13.656671 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:13.731460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1355/2726] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:17:13 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:17:13.735551 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520e54a-3990-4275-bde0-5db5194639e4 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:13.746188 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520e54a-3990-4275-bde0-5db5194639e4 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:13.751050 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c217e9f-a350-41b3-9169-9815e908e289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:13.751651 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c217e9f-a350-41b3-9169-9815e908e289 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:13.752165 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c217e9f-a350-41b3-9169-9815e908e289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:13.752467 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c217e9f-a350-41b3-9169-9815e908e289 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:13.753272 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c217e9f-a350-41b3-9169-9815e908e289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:13.755390 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c217e9f-a350-41b3-9169-9815e908e289 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:13.765993 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1372/2727] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:17:13.772790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543860cd-81ab-480b-ae01-f1ec3823fcf5 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:13.782782 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543860cd-81ab-480b-ae01-f1ec3823fcf5 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:13.787357 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9144cf-3870-4a15-a22f-c797c5876af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:13.787945 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9144cf-3870-4a15-a22f-c797c5876af2 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:13.788264 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9144cf-3870-4a15-a22f-c797c5876af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:13.788538 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9144cf-3870-4a15-a22f-c797c5876af2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:13.789175 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9144cf-3870-4a15-a22f-c797c5876af2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:13.790814 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9144cf-3870-4a15-a22f-c797c5876af2 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:13.801333 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1356/2728] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:17:13.805102 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c217e9f-a350-41b3-9169-9815e908e289 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:13.815032 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c217e9f-a350-41b3-9169-9815e908e289 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:13.820171 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65e886-cf9c-47e7-9f86-3462aa1f7ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:13.820813 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65e886-cf9c-47e7-9f86-3462aa1f7ee7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:13.821093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65e886-cf9c-47e7-9f86-3462aa1f7ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:13.821366 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65e886-cf9c-47e7-9f86-3462aa1f7ee7 admin admin] PATH_INFO: `/v3/projects/1a518e979c944b3ab2a1245b3cfd246f/users/56e50c46ac774417a50af986c5f87420/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:13.837150 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f65e886-cf9c-47e7-9f86-3462aa1f7ee7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a518e979c944b3ab2a1245b3cfd246f, user_id=56e50c46ac774417a50af986c5f87420, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:13.839753 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f65e886-cf9c-47e7-9f86-3462aa1f7ee7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:13.885958 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1373/2729] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:17:13 2022] PUT /identity/v3/projects/1a518e979c944b3ab2a1245b3cfd246f/users/56e50c46ac774417a50af986c5f87420/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:17:13.890848 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ee6c6-0147-413c-bf81-a688b340605b None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:13.891135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ee6c6-0147-413c-bf81-a688b340605b None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:13.891477 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ee6c6-0147-413c-bf81-a688b340605b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:13.911043 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-643ee6c6-0147-413c-bf81-a688b340605b None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:13.912286 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-643ee6c6-0147-413c-bf81-a688b340605b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:13.972327 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1357/2730] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:17:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:17:13.981895 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65e886-cf9c-47e7-9f86-3462aa1f7ee7 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:13.983761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5f65e886-cf9c-47e7-9f86-3462aa1f7ee7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:14.040681 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65e886-cf9c-47e7-9f86-3462aa1f7ee7 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:14.044232 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c253dba-16f2-4852-8f92-3b8c38921ff6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:14.044850 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c253dba-16f2-4852-8f92-3b8c38921ff6 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:14.045125 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c253dba-16f2-4852-8f92-3b8c38921ff6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:14.045394 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c253dba-16f2-4852-8f92-3b8c38921ff6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:14.054631 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c253dba-16f2-4852-8f92-3b8c38921ff6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:14.056042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c253dba-16f2-4852-8f92-3b8c38921ff6 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:14.065198 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1374/2731] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:17:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:17:18.945123 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc90dfe-fc6b-43b8-a286-da9f8be9b88d None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:18.945664 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc90dfe-fc6b-43b8-a286-da9f8be9b88d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:18.945792 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc90dfe-fc6b-43b8-a286-da9f8be9b88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:18.989304 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-4dc90dfe-fc6b-43b8-a286-da9f8be9b88d None None] MFA Rules not processed for user `56e50c46ac774417a50af986c5f87420`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:18.989812 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-4dc90dfe-fc6b-43b8-a286-da9f8be9b88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:19.095942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1358/2732] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:17:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:17:19.110093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c253dba-16f2-4852-8f92-3b8c38921ff6 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:19.111670 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9c253dba-16f2-4852-8f92-3b8c38921ff6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:19.161227 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:19.196624 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c253dba-16f2-4852-8f92-3b8c38921ff6 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:19.202305 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f60cfa-7e3a-4f24-b413-a7fe3e236e02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:19.202910 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f60cfa-7e3a-4f24-b413-a7fe3e236e02 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:19.203190 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f60cfa-7e3a-4f24-b413-a7fe3e236e02 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:19.203463 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f60cfa-7e3a-4f24-b413-a7fe3e236e02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:19.213667 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f60cfa-7e3a-4f24-b413-a7fe3e236e02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:19.215282 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f60cfa-7e3a-4f24-b413-a7fe3e236e02 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:19.225224 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1375/2733] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:17:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:17:21.643928 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415fee50-1cd6-4e76-8e6e-0f92d25bf5f8 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:21.644471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415fee50-1cd6-4e76-8e6e-0f92d25bf5f8 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:21.644720 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415fee50-1cd6-4e76-8e6e-0f92d25bf5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:21.664736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-415fee50-1cd6-4e76-8e6e-0f92d25bf5f8 None None] MFA Rules not processed for user `56e50c46ac774417a50af986c5f87420`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:21.665948 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-415fee50-1cd6-4e76-8e6e-0f92d25bf5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:21.672858 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1359/2734] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:17:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:17:26.467798 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f60cfa-7e3a-4f24-b413-a7fe3e236e02 service nova] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:26.480332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f60cfa-7e3a-4f24-b413-a7fe3e236e02 service nova] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:26.485914 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e3b619-3cc3-4819-85ed-cd7aa410a941 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:26.486507 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3b619-3cc3-4819-85ed-cd7aa410a941 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:26.486782 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3b619-3cc3-4819-85ed-cd7aa410a941 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:26.487053 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3b619-3cc3-4819-85ed-cd7aa410a941 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:26.489360 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c2e3b619-3cc3-4819-85ed-cd7aa410a941 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:26.501619 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e3b619-3cc3-4819-85ed-cd7aa410a941 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:26.503211 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e3b619-3cc3-4819-85ed-cd7aa410a941 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:26.514142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1376/2735] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:17:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:17:31.658610 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:INSERT=3,keystone:SELECT=69 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:36.162318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415fee50-1cd6-4e76-8e6e-0f92d25bf5f8 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:36.163847 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-415fee50-1cd6-4e76-8e6e-0f92d25bf5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:36.177731 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415fee50-1cd6-4e76-8e6e-0f92d25bf5f8 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:36.182190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10471a9a-1d40-4b30-8d79-9d6d864bda8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:36.182806 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10471a9a-1d40-4b30-8d79-9d6d864bda8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:36.183084 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10471a9a-1d40-4b30-8d79-9d6d864bda8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:36.183355 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10471a9a-1d40-4b30-8d79-9d6d864bda8c admin admin] PATH_INFO: `/v3/users/35ee93a615ff4485a500f99d7fbb273d` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:36.184521 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10471a9a-1d40-4b30-8d79-9d6d864bda8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=35ee93a615ff4485a500f99d7fbb273d)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:36.186435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10471a9a-1d40-4b30-8d79-9d6d864bda8c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:36.368321 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-10471a9a-1d40-4b30-8d79-9d6d864bda8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35ee93a615ff4485a500f99d7fbb273d'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:36.368979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-10471a9a-1d40-4b30-8d79-9d6d864bda8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35ee93a615ff4485a500f99d7fbb273d'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:36.376158 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-10471a9a-1d40-4b30-8d79-9d6d864bda8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35ee93a615ff4485a500f99d7fbb273d'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:36.387953 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-10471a9a-1d40-4b30-8d79-9d6d864bda8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35ee93a615ff4485a500f99d7fbb273d'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:36.388632 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-10471a9a-1d40-4b30-8d79-9d6d864bda8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35ee93a615ff4485a500f99d7fbb273d'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:36.456947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1360/2736] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:17:36 2022] DELETE /identity/v3/users/35ee93a615ff4485a500f99d7fbb273d => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:17:36.462005 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0dcb7-83bd-4368-873d-9a0d18ef1e3a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:36.462491 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0dcb7-83bd-4368-873d-9a0d18ef1e3a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:36.462610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0dcb7-83bd-4368-873d-9a0d18ef1e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:36.481596 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a4e0dcb7-83bd-4368-873d-9a0d18ef1e3a None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:36.482781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a4e0dcb7-83bd-4368-873d-9a0d18ef1e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:36.539798 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1377/2737] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:17:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:17:36.549727 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10471a9a-1d40-4b30-8d79-9d6d864bda8c admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:36.551132 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-10471a9a-1d40-4b30-8d79-9d6d864bda8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:36.609578 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10471a9a-1d40-4b30-8d79-9d6d864bda8c admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:36.613275 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7782b6-788a-4c20-90d9-c0b66afc59fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:36.613884 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7782b6-788a-4c20-90d9-c0b66afc59fa service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:36.614163 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7782b6-788a-4c20-90d9-c0b66afc59fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:36.614433 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7782b6-788a-4c20-90d9-c0b66afc59fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:36.623494 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7782b6-788a-4c20-90d9-c0b66afc59fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:36.624964 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7782b6-788a-4c20-90d9-c0b66afc59fa service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:36.636404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1361/2738] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:17:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:17:37.041747 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e0dcb7-83bd-4368-873d-9a0d18ef1e3a None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:37.045553 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a4e0dcb7-83bd-4368-873d-9a0d18ef1e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:37.056599 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e0dcb7-83bd-4368-873d-9a0d18ef1e3a None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:37.061227 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c13cc-89c9-4d78-a358-827fea2eb010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:37.061903 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c13cc-89c9-4d78-a358-827fea2eb010 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:37.062187 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c13cc-89c9-4d78-a358-827fea2eb010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:37.062517 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c13cc-89c9-4d78-a358-827fea2eb010 admin admin] PATH_INFO: `/v3/projects/536bf5a46b4d4ca69ddf05a2f617ee9e` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:37.063829 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c13cc-89c9-4d78-a358-827fea2eb010 admin admin] RBAC: Authorizing `identity:delete_project(project_id=536bf5a46b4d4ca69ddf05a2f617ee9e)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:37.065665 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c13cc-89c9-4d78-a358-827fea2eb010 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:37.154436 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d97c13cc-89c9-4d78-a358-827fea2eb010 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '536bf5a46b4d4ca69ddf05a2f617ee9e', 'request_id': 'req-d97c13cc-89c9-4d78-a358-827fea2eb010'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:37.160652 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d97c13cc-89c9-4d78-a358-827fea2eb010 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '536bf5a46b4d4ca69ddf05a2f617ee9e', 'request_id': 'req-d97c13cc-89c9-4d78-a358-827fea2eb010'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:37.222849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d97c13cc-89c9-4d78-a358-827fea2eb010 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '536bf5a46b4d4ca69ddf05a2f617ee9e', 'request_id': 'req-d97c13cc-89c9-4d78-a358-827fea2eb010'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:37.233283 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d97c13cc-89c9-4d78-a358-827fea2eb010 admin admin] The token cache is being invalidate because project 536bf5a46b4d4ca69ddf05a2f617ee9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:17:37.233574 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d97c13cc-89c9-4d78-a358-827fea2eb010 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:37.234199 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d97c13cc-89c9-4d78-a358-827fea2eb010 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:37.235666 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1378/2739] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:17:37 2022] DELETE /identity/v3/projects/536bf5a46b4d4ca69ddf05a2f617ee9e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:17:37.240304 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7782b6-788a-4c20-90d9-c0b66afc59fa service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:37.249882 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7782b6-788a-4c20-90d9-c0b66afc59fa service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:37.305021 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b533ad0a-b980-4fbf-b374-f032b27b438c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:37.305650 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533ad0a-b980-4fbf-b374-f032b27b438c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:37.305924 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533ad0a-b980-4fbf-b374-f032b27b438c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:37.306196 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533ad0a-b980-4fbf-b374-f032b27b438c admin admin] PATH_INFO: `/v3/users/6aaea8d20e2241f88ebba26a853b9d76` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:37.307353 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b533ad0a-b980-4fbf-b374-f032b27b438c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aaea8d20e2241f88ebba26a853b9d76)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:37.309291 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b533ad0a-b980-4fbf-b374-f032b27b438c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:37.325062 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:37.493733 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b533ad0a-b980-4fbf-b374-f032b27b438c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aaea8d20e2241f88ebba26a853b9d76'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:37.494340 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b533ad0a-b980-4fbf-b374-f032b27b438c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aaea8d20e2241f88ebba26a853b9d76'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:37.501448 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b533ad0a-b980-4fbf-b374-f032b27b438c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aaea8d20e2241f88ebba26a853b9d76'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:37.514241 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b533ad0a-b980-4fbf-b374-f032b27b438c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aaea8d20e2241f88ebba26a853b9d76'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:37.518414 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b533ad0a-b980-4fbf-b374-f032b27b438c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aaea8d20e2241f88ebba26a853b9d76'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:37.620818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1362/2740] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:17:37 2022] DELETE /identity/v3/users/6aaea8d20e2241f88ebba26a853b9d76 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:17:38.079975 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c13cc-89c9-4d78-a358-827fea2eb010 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:38.082547 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d97c13cc-89c9-4d78-a358-827fea2eb010 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:38.105111 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c13cc-89c9-4d78-a358-827fea2eb010 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:38.134663 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:38.162615 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc048db8-7294-4004-8933-afbdc65145a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:38.163281 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc048db8-7294-4004-8933-afbdc65145a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:38.163752 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc048db8-7294-4004-8933-afbdc65145a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:38.164096 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc048db8-7294-4004-8933-afbdc65145a6 admin admin] PATH_INFO: `/v3/projects/fb1d56b893cf45729d68dcb5fef86dc5` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:38.165685 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc048db8-7294-4004-8933-afbdc65145a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb1d56b893cf45729d68dcb5fef86dc5)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:38.167723 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc048db8-7294-4004-8933-afbdc65145a6 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:38.266974 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fc048db8-7294-4004-8933-afbdc65145a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb1d56b893cf45729d68dcb5fef86dc5', 'request_id': 'req-fc048db8-7294-4004-8933-afbdc65145a6'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:38.273703 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fc048db8-7294-4004-8933-afbdc65145a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb1d56b893cf45729d68dcb5fef86dc5', 'request_id': 'req-fc048db8-7294-4004-8933-afbdc65145a6'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:38.335504 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fc048db8-7294-4004-8933-afbdc65145a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb1d56b893cf45729d68dcb5fef86dc5', 'request_id': 'req-fc048db8-7294-4004-8933-afbdc65145a6'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:38.346109 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fc048db8-7294-4004-8933-afbdc65145a6 admin admin] The token cache is being invalidate because project fb1d56b893cf45729d68dcb5fef86dc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:17:38.346401 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fc048db8-7294-4004-8933-afbdc65145a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:38.347020 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fc048db8-7294-4004-8933-afbdc65145a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:38.348546 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1379/2741] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:17:38 2022] DELETE /identity/v3/projects/fb1d56b893cf45729d68dcb5fef86dc5 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:17:38.353426 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb7bd4-4019-44c5-859e-bec634152a8c None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:38.353912 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb7bd4-4019-44c5-859e-bec634152a8c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:38.354038 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb7bd4-4019-44c5-859e-bec634152a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:38.373134 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-99cb7bd4-4019-44c5-859e-bec634152a8c None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:38.374340 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-99cb7bd4-4019-44c5-859e-bec634152a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:38.432516 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1363/2742] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:17:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:17:38.440355 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbfe568-4fc9-4e2c-93a0-392f58982170 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:38.440645 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbfe568-4fc9-4e2c-93a0-392f58982170 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:38.440925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbfe568-4fc9-4e2c-93a0-392f58982170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:38.461510 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-abbfe568-4fc9-4e2c-93a0-392f58982170 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:38.462714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-abbfe568-4fc9-4e2c-93a0-392f58982170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:38.470451 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1380/2743] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:17:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:17:38.474598 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb7bd4-4019-44c5-859e-bec634152a8c None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:38.484474 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb7bd4-4019-44c5-859e-bec634152a8c None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:38.489307 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63e050-cc9e-4649-bbde-81eb82c54b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:38.489920 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63e050-cc9e-4649-bbde-81eb82c54b0d admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:38.490255 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63e050-cc9e-4649-bbde-81eb82c54b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:38.490524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63e050-cc9e-4649-bbde-81eb82c54b0d admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:38.491168 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63e050-cc9e-4649-bbde-81eb82c54b0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:17:38.491498 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63e050-cc9e-4649-bbde-81eb82c54b0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:38.493118 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63e050-cc9e-4649-bbde-81eb82c54b0d admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:38.504949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1364/2744] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:17:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:17:38.509243 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbfe568-4fc9-4e2c-93a0-392f58982170 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:38.518535 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbfe568-4fc9-4e2c-93a0-392f58982170 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:38.523523 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7030d8-8e0d-47c1-b09a-6a393bdbfef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:38.524155 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7030d8-8e0d-47c1-b09a-6a393bdbfef2 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:38.524434 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7030d8-8e0d-47c1-b09a-6a393bdbfef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:38.524704 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7030d8-8e0d-47c1-b09a-6a393bdbfef2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:38.525381 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7030d8-8e0d-47c1-b09a-6a393bdbfef2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:38.527248 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7030d8-8e0d-47c1-b09a-6a393bdbfef2 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:38.630404 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1381/2745] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:17:38 2022] POST /identity/v3/projects => generated 361 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:17:38.634672 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63e050-cc9e-4649-bbde-81eb82c54b0d admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:38.645667 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63e050-cc9e-4649-bbde-81eb82c54b0d admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:38.711208 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a3c55-aef6-4721-adce-5ea6bb52d47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:38.711815 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a3c55-aef6-4721-adce-5ea6bb52d47c admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:38.712120 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a3c55-aef6-4721-adce-5ea6bb52d47c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:38.712402 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a3c55-aef6-4721-adce-5ea6bb52d47c admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:38.713043 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a3c55-aef6-4721-adce-5ea6bb52d47c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:38.715097 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a3c55-aef6-4721-adce-5ea6bb52d47c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:38.802358 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1365/2746] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:17:38 2022] POST /identity/v3/users => generated 281 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:17:38.806505 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7030d8-8e0d-47c1-b09a-6a393bdbfef2 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:38.817843 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7030d8-8e0d-47c1-b09a-6a393bdbfef2 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:38.822718 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21fef5-1fdd-45bf-9f51-1e11f575d055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:38.823681 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21fef5-1fdd-45bf-9f51-1e11f575d055 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:38.824220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21fef5-1fdd-45bf-9f51-1e11f575d055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:38.824686 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21fef5-1fdd-45bf-9f51-1e11f575d055 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:38.825350 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb21fef5-1fdd-45bf-9f51-1e11f575d055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:38.826976 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb21fef5-1fdd-45bf-9f51-1e11f575d055 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:38.839965 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1382/2747] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:17:38.843970 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a3c55-aef6-4721-adce-5ea6bb52d47c admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:38.854040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a3c55-aef6-4721-adce-5ea6bb52d47c admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:38.860896 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d5b7f4-ddbd-44e9-84df-f57c6fcd5fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:38.861932 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5b7f4-ddbd-44e9-84df-f57c6fcd5fcf admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:38.862497 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5b7f4-ddbd-44e9-84df-f57c6fcd5fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:38.862782 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5b7f4-ddbd-44e9-84df-f57c6fcd5fcf admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:38.863951 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d5b7f4-ddbd-44e9-84df-f57c6fcd5fcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:38.865558 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d5b7f4-ddbd-44e9-84df-f57c6fcd5fcf admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:38.871004 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:38.876750 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1366/2748] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:17:38.884091 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21fef5-1fdd-45bf-9f51-1e11f575d055 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:38.894263 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21fef5-1fdd-45bf-9f51-1e11f575d055 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:38.898877 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7bf7e-9fac-473f-84df-0267ddd7a2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:38.899529 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7bf7e-9fac-473f-84df-0267ddd7a2ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:38.899920 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7bf7e-9fac-473f-84df-0267ddd7a2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:38.900233 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7bf7e-9fac-473f-84df-0267ddd7a2ac admin admin] PATH_INFO: `/v3/projects/f4558c156b9d4d478e1b20c00646e975/users/d0b737e7f7f7455d8ef1f8d16c0585d0/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:38.916623 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7bf7e-9fac-473f-84df-0267ddd7a2ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4558c156b9d4d478e1b20c00646e975, user_id=d0b737e7f7f7455d8ef1f8d16c0585d0, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:38.919266 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7bf7e-9fac-473f-84df-0267ddd7a2ac admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:38.991661 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1383/2749] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:17:38 2022] PUT /identity/v3/projects/f4558c156b9d4d478e1b20c00646e975/users/d0b737e7f7f7455d8ef1f8d16c0585d0/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:17:39.004871 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29e542-5171-4f1d-b54a-f0de1cdfd10c None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:39.005172 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29e542-5171-4f1d-b54a-f0de1cdfd10c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:39.005419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29e542-5171-4f1d-b54a-f0de1cdfd10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:39.036030 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-5b29e542-5171-4f1d-b54a-f0de1cdfd10c None None] MFA Rules not processed for user `d0b737e7f7f7455d8ef1f8d16c0585d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:39.037311 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5b29e542-5171-4f1d-b54a-f0de1cdfd10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:39.104512 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1367/2750] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:17:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:17:39.139931 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7bf7e-9fac-473f-84df-0267ddd7a2ac admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:39.141674 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-95c7bf7e-9fac-473f-84df-0267ddd7a2ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:39.202966 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7bf7e-9fac-473f-84df-0267ddd7a2ac admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:39.207291 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b52e2-cdb4-4ca7-8056-dffc06b62fea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:39.208430 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b52e2-cdb4-4ca7-8056-dffc06b62fea service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:39.208915 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b52e2-cdb4-4ca7-8056-dffc06b62fea service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:39.209377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b52e2-cdb4-4ca7-8056-dffc06b62fea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:39.220365 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b52e2-cdb4-4ca7-8056-dffc06b62fea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:39.223058 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b52e2-cdb4-4ca7-8056-dffc06b62fea service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:39.227833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:39.232914 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1384/2751] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:17:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:17:39.249277 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29e542-5171-4f1d-b54a-f0de1cdfd10c None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:39.319328 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29e542-5171-4f1d-b54a-f0de1cdfd10c None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:39.323035 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a345b95-30b1-4925-8d26-573cbdb643a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:39.323679 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a345b95-30b1-4925-8d26-573cbdb643a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:39.323956 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a345b95-30b1-4925-8d26-573cbdb643a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:39.324302 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a345b95-30b1-4925-8d26-573cbdb643a2 admin admin] PATH_INFO: `/v3/users/d0b737e7f7f7455d8ef1f8d16c0585d0` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:39.325442 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a345b95-30b1-4925-8d26-573cbdb643a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0b737e7f7f7455d8ef1f8d16c0585d0)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:39.327356 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a345b95-30b1-4925-8d26-573cbdb643a2 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:39.478132 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3a345b95-30b1-4925-8d26-573cbdb643a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0b737e7f7f7455d8ef1f8d16c0585d0'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:39.478994 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3a345b95-30b1-4925-8d26-573cbdb643a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0b737e7f7f7455d8ef1f8d16c0585d0'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:39.487170 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3a345b95-30b1-4925-8d26-573cbdb643a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0b737e7f7f7455d8ef1f8d16c0585d0'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:39.499911 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3a345b95-30b1-4925-8d26-573cbdb643a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0b737e7f7f7455d8ef1f8d16c0585d0'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:39.500725 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3a345b95-30b1-4925-8d26-573cbdb643a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0b737e7f7f7455d8ef1f8d16c0585d0'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:39.571575 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1368/2752] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:17:39 2022] DELETE /identity/v3/users/d0b737e7f7f7455d8ef1f8d16c0585d0 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:17:39.576129 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674c3c2-f12f-459b-b4a3-a32a67e8176c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:39.581605 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674c3c2-f12f-459b-b4a3-a32a67e8176c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:39.581605 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674c3c2-f12f-459b-b4a3-a32a67e8176c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:39.596604 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-2674c3c2-f12f-459b-b4a3-a32a67e8176c None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:39.598439 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2674c3c2-f12f-459b-b4a3-a32a67e8176c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:39.657791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1385/2753] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:17:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:17:39.668200 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a345b95-30b1-4925-8d26-573cbdb643a2 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:39.673282 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3a345b95-30b1-4925-8d26-573cbdb643a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:39.680435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:39.727041 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a345b95-30b1-4925-8d26-573cbdb643a2 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:39.730647 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4b34f-d4d3-4cb9-a375-cd4918c9e4f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:39.731288 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4b34f-d4d3-4cb9-a375-cd4918c9e4f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:39.731566 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4b34f-d4d3-4cb9-a375-cd4918c9e4f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:39.731834 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4b34f-d4d3-4cb9-a375-cd4918c9e4f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:39.741283 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae4b34f-d4d3-4cb9-a375-cd4918c9e4f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:39.743758 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae4b34f-d4d3-4cb9-a375-cd4918c9e4f5 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:39.753681 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1369/2754] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:17:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:17:40.197138 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2674c3c2-f12f-459b-b4a3-a32a67e8176c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:40.198824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2674c3c2-f12f-459b-b4a3-a32a67e8176c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:40.216312 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2674c3c2-f12f-459b-b4a3-a32a67e8176c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:40.221432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a7d2f-a46a-4259-a031-911c49b80402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:40.222144 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a7d2f-a46a-4259-a031-911c49b80402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:40.222429 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a7d2f-a46a-4259-a031-911c49b80402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:40.222711 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a7d2f-a46a-4259-a031-911c49b80402 admin admin] PATH_INFO: `/v3/projects/f4558c156b9d4d478e1b20c00646e975` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:40.223893 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0a7d2f-a46a-4259-a031-911c49b80402 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4558c156b9d4d478e1b20c00646e975)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:40.226033 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0a7d2f-a46a-4259-a031-911c49b80402 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:40.365708 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5c0a7d2f-a46a-4259-a031-911c49b80402 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4558c156b9d4d478e1b20c00646e975', 'request_id': 'req-5c0a7d2f-a46a-4259-a031-911c49b80402'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:40.372244 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5c0a7d2f-a46a-4259-a031-911c49b80402 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4558c156b9d4d478e1b20c00646e975', 'request_id': 'req-5c0a7d2f-a46a-4259-a031-911c49b80402'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:40.479260 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5c0a7d2f-a46a-4259-a031-911c49b80402 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4558c156b9d4d478e1b20c00646e975', 'request_id': 'req-5c0a7d2f-a46a-4259-a031-911c49b80402'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:40.486150 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4b34f-d4d3-4cb9-a375-cd4918c9e4f5 service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:40.487899 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-bae4b34f-d4d3-4cb9-a375-cd4918c9e4f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:40.493613 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5c0a7d2f-a46a-4259-a031-911c49b80402 admin admin] The token cache is being invalidate because project f4558c156b9d4d478e1b20c00646e975 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:17:40.493905 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5c0a7d2f-a46a-4259-a031-911c49b80402 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:40.494620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5c0a7d2f-a46a-4259-a031-911c49b80402 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:40.496432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1386/2755] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:17:40 2022] DELETE /identity/v3/projects/f4558c156b9d4d478e1b20c00646e975 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:17:40.501735 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e5c06-855c-4ba7-8df9-03e69b6aba22 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:40.502142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e5c06-855c-4ba7-8df9-03e69b6aba22 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:40.502565 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e5c06-855c-4ba7-8df9-03e69b6aba22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:40.529731 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-df5e5c06-855c-4ba7-8df9-03e69b6aba22 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:40.531189 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-df5e5c06-855c-4ba7-8df9-03e69b6aba22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:40.569758 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4b34f-d4d3-4cb9-a375-cd4918c9e4f5 service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:40.601197 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1387/2756] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:17:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:17:40.608641 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169262b7-82af-42db-a8d3-42f6aec28fd7 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:40.609097 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169262b7-82af-42db-a8d3-42f6aec28fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:40.609376 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169262b7-82af-42db-a8d3-42f6aec28fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:40.636012 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-169262b7-82af-42db-a8d3-42f6aec28fd7 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:40.636012 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-169262b7-82af-42db-a8d3-42f6aec28fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:40.643054 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1388/2757] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:17:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:17:40.644117 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950147ce-f450-4a45-9886-37369673dfa5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:40.644281 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169262b7-82af-42db-a8d3-42f6aec28fd7 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:40.645199 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950147ce-f450-4a45-9886-37369673dfa5 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:40.645631 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950147ce-f450-4a45-9886-37369673dfa5 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:40.645906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950147ce-f450-4a45-9886-37369673dfa5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:40.647675 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-950147ce-f450-4a45-9886-37369673dfa5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:40.653433 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169262b7-82af-42db-a8d3-42f6aec28fd7 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:40.658133 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d98a48-6a22-4d7e-9caa-dfd4f099b0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:40.658732 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d98a48-6a22-4d7e-9caa-dfd4f099b0ae admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:40.659012 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d98a48-6a22-4d7e-9caa-dfd4f099b0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:40.659290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d98a48-6a22-4d7e-9caa-dfd4f099b0ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:40.660485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d98a48-6a22-4d7e-9caa-dfd4f099b0ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:17:40.665009 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d98a48-6a22-4d7e-9caa-dfd4f099b0ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:40.665147 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950147ce-f450-4a45-9886-37369673dfa5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:40.665282 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d98a48-6a22-4d7e-9caa-dfd4f099b0ae admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:40.665430 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950147ce-f450-4a45-9886-37369673dfa5 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:40.679591 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1389/2758] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:17:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:17:40.680679 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d98a48-6a22-4d7e-9caa-dfd4f099b0ae admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:40.684336 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1370/2759] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:17:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:17:40.687000 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:40.693580 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d98a48-6a22-4d7e-9caa-dfd4f099b0ae admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:40.695771 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126e776-1a9a-482e-af5b-80dc3dea1bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:40.696415 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126e776-1a9a-482e-af5b-80dc3dea1bb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:40.696701 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126e776-1a9a-482e-af5b-80dc3dea1bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:40.696976 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126e776-1a9a-482e-af5b-80dc3dea1bb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:40.697610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c126e776-1a9a-482e-af5b-80dc3dea1bb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:40.699677 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c126e776-1a9a-482e-af5b-80dc3dea1bb2 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:40.813199 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1390/2760] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:17:40 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:17:40.813748 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950147ce-f450-4a45-9886-37369673dfa5 service nova] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:40.827174 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950147ce-f450-4a45-9886-37369673dfa5 service nova] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:40.865136 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:40.897441 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95769b1f-4d7c-493e-9b59-741b28418f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:40.898250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95769b1f-4d7c-493e-9b59-741b28418f78 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:40.898629 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95769b1f-4d7c-493e-9b59-741b28418f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:40.899006 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95769b1f-4d7c-493e-9b59-741b28418f78 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:40.899818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95769b1f-4d7c-493e-9b59-741b28418f78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:40.902168 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95769b1f-4d7c-493e-9b59-741b28418f78 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:41.000326 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1371/2761] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:17:40 2022] POST /identity/v3/users => generated 292 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:17:41.004419 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126e776-1a9a-482e-af5b-80dc3dea1bb2 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:41.022661 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126e776-1a9a-482e-af5b-80dc3dea1bb2 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:41.029815 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913dc6f8-67c4-48a5-a7a2-f9effba4e6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:41.030550 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913dc6f8-67c4-48a5-a7a2-f9effba4e6e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:41.031442 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913dc6f8-67c4-48a5-a7a2-f9effba4e6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:41.031713 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913dc6f8-67c4-48a5-a7a2-f9effba4e6e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:41.032496 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913dc6f8-67c4-48a5-a7a2-f9effba4e6e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:41.034737 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913dc6f8-67c4-48a5-a7a2-f9effba4e6e0 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:41.046475 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1391/2762] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:17:41.050322 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95769b1f-4d7c-493e-9b59-741b28418f78 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:41.062183 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95769b1f-4d7c-493e-9b59-741b28418f78 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:41.068089 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e9eaef-5fc4-4c7d-841f-95520c0f6e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:41.068863 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9eaef-5fc4-4c7d-841f-95520c0f6e30 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:41.069638 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9eaef-5fc4-4c7d-841f-95520c0f6e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:41.070179 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9eaef-5fc4-4c7d-841f-95520c0f6e30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:41.070995 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e9eaef-5fc4-4c7d-841f-95520c0f6e30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:41.072825 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e9eaef-5fc4-4c7d-841f-95520c0f6e30 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:41.085185 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1372/2763] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:17:41.089789 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913dc6f8-67c4-48a5-a7a2-f9effba4e6e0 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:41.101153 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913dc6f8-67c4-48a5-a7a2-f9effba4e6e0 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:41.106430 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c270c1f7-9422-4bdc-9ec8-8a864b432af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:41.107359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270c1f7-9422-4bdc-9ec8-8a864b432af5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:41.107926 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270c1f7-9422-4bdc-9ec8-8a864b432af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:41.108098 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270c1f7-9422-4bdc-9ec8-8a864b432af5 admin admin] PATH_INFO: `/v3/projects/772b42b45d054104ab93ceb8a2cc4a9d/users/ab66b0f97a0e4b49a59377fa4dac5111/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:41.123753 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c270c1f7-9422-4bdc-9ec8-8a864b432af5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=772b42b45d054104ab93ceb8a2cc4a9d, user_id=ab66b0f97a0e4b49a59377fa4dac5111, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:41.126609 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c270c1f7-9422-4bdc-9ec8-8a864b432af5 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:41.190563 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1392/2764] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:17:41 2022] PUT /identity/v3/projects/772b42b45d054104ab93ceb8a2cc4a9d/users/ab66b0f97a0e4b49a59377fa4dac5111/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:17:41.200829 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833b49b-15d6-47b5-9395-d9744f9cc28b None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:41.201311 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833b49b-15d6-47b5-9395-d9744f9cc28b None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:41.201716 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833b49b-15d6-47b5-9395-d9744f9cc28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:41.234047 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-e833b49b-15d6-47b5-9395-d9744f9cc28b None None] MFA Rules not processed for user `ab66b0f97a0e4b49a59377fa4dac5111`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:41.235280 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e833b49b-15d6-47b5-9395-d9744f9cc28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:41.307133 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1373/2765] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:17:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:17:41.311952 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c270c1f7-9422-4bdc-9ec8-8a864b432af5 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:41.376439 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c270c1f7-9422-4bdc-9ec8-8a864b432af5 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:41.382647 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d353f2-cda0-400d-957f-fc006dcfe9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:41.383282 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d353f2-cda0-400d-957f-fc006dcfe9a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:41.383913 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d353f2-cda0-400d-957f-fc006dcfe9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:41.384296 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d353f2-cda0-400d-957f-fc006dcfe9a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:41.384947 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d353f2-cda0-400d-957f-fc006dcfe9a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:41.386993 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d353f2-cda0-400d-957f-fc006dcfe9a4 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:41.464155 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1393/2766] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:17:41 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:17:41.468178 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833b49b-15d6-47b5-9395-d9744f9cc28b None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:41.479394 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833b49b-15d6-47b5-9395-d9744f9cc28b None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:41.540762 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5ac74-2c56-4989-a937-8dd573093025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:41.541495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5ac74-2c56-4989-a937-8dd573093025 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:41.541629 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5ac74-2c56-4989-a937-8dd573093025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:41.541898 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5ac74-2c56-4989-a937-8dd573093025 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:41.542551 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e5ac74-2c56-4989-a937-8dd573093025 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:41.544504 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e5ac74-2c56-4989-a937-8dd573093025 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:41.617233 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1374/2767] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:17:41 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:17:41.621430 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d353f2-cda0-400d-957f-fc006dcfe9a4 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:41.632365 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d353f2-cda0-400d-957f-fc006dcfe9a4 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:41.637200 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df12627-7b5c-4f94-8ddc-51340c8fac2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:41.637796 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df12627-7b5c-4f94-8ddc-51340c8fac2f admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:41.638074 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df12627-7b5c-4f94-8ddc-51340c8fac2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:41.638337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df12627-7b5c-4f94-8ddc-51340c8fac2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:41.638979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df12627-7b5c-4f94-8ddc-51340c8fac2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:41.640688 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df12627-7b5c-4f94-8ddc-51340c8fac2f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:41.652802 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1394/2768] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:17:41.656802 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5ac74-2c56-4989-a937-8dd573093025 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:41.666637 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5ac74-2c56-4989-a937-8dd573093025 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:41.671211 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52046ea8-147f-4232-b359-35bc7d3aa3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:41.674873 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52046ea8-147f-4232-b359-35bc7d3aa3f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:41.674873 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52046ea8-147f-4232-b359-35bc7d3aa3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:41.674873 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52046ea8-147f-4232-b359-35bc7d3aa3f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:41.674873 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52046ea8-147f-4232-b359-35bc7d3aa3f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:41.674873 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52046ea8-147f-4232-b359-35bc7d3aa3f1 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:41.684746 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1375/2769] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:17:41.688743 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df12627-7b5c-4f94-8ddc-51340c8fac2f admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:41.700700 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df12627-7b5c-4f94-8ddc-51340c8fac2f admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:41.705543 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67606ee5-1805-49ae-a254-361093072d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:41.706465 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67606ee5-1805-49ae-a254-361093072d6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:41.706791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67606ee5-1805-49ae-a254-361093072d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:41.707010 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67606ee5-1805-49ae-a254-361093072d6e admin admin] PATH_INFO: `/v3/projects/8219f46e43e7468ba707ce9a60ac3875/users/74b2855f803949959d1fce08f6a48880/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:41.722886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67606ee5-1805-49ae-a254-361093072d6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8219f46e43e7468ba707ce9a60ac3875, user_id=74b2855f803949959d1fce08f6a48880, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:41.725672 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67606ee5-1805-49ae-a254-361093072d6e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:41.786579 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1395/2770] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:17:41 2022] PUT /identity/v3/projects/8219f46e43e7468ba707ce9a60ac3875/users/74b2855f803949959d1fce08f6a48880/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:17:41.797869 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb59f9a-66e4-4b2e-b606-31922e1db067 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:41.798150 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb59f9a-66e4-4b2e-b606-31922e1db067 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:41.798421 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb59f9a-66e4-4b2e-b606-31922e1db067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:41.828189 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-8eb59f9a-66e4-4b2e-b606-31922e1db067 None None] MFA Rules not processed for user `74b2855f803949959d1fce08f6a48880`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:41.829400 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8eb59f9a-66e4-4b2e-b606-31922e1db067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:41.838750 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:41.893373 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1376/2771] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:17:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:17:41.897856 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67606ee5-1805-49ae-a254-361093072d6e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:41.954894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:41.958783 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67606ee5-1805-49ae-a254-361093072d6e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:41.963587 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002539a3-77bc-4ace-8888-ae15f84a6d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:41.967219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002539a3-77bc-4ace-8888-ae15f84a6d7c admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:41.967219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002539a3-77bc-4ace-8888-ae15f84a6d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:41.967219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002539a3-77bc-4ace-8888-ae15f84a6d7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:41.967219 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002539a3-77bc-4ace-8888-ae15f84a6d7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:41.967607 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002539a3-77bc-4ace-8888-ae15f84a6d7c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:42.081435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1396/2772] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:17:41 2022] POST /identity/v3/projects => generated 385 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:17:42.085518 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb59f9a-66e4-4b2e-b606-31922e1db067 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:42.096231 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb59f9a-66e4-4b2e-b606-31922e1db067 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:42.151585 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a7be8-27f7-4e7e-8f9a-378b46242739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:42.152213 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a7be8-27f7-4e7e-8f9a-378b46242739 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:42.152489 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a7be8-27f7-4e7e-8f9a-378b46242739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:42.152751 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a7be8-27f7-4e7e-8f9a-378b46242739 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:42.153365 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a7be8-27f7-4e7e-8f9a-378b46242739 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:42.155259 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a7be8-27f7-4e7e-8f9a-378b46242739 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:42.257942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1377/2773] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:17:42 2022] POST /identity/v3/users => generated 299 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:17:42.261980 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002539a3-77bc-4ace-8888-ae15f84a6d7c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:42.272367 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002539a3-77bc-4ace-8888-ae15f84a6d7c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:42.277001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280586a3-9a4c-4e57-af7f-c03e44c37ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:42.277594 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280586a3-9a4c-4e57-af7f-c03e44c37ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:42.277906 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280586a3-9a4c-4e57-af7f-c03e44c37ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:42.278183 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280586a3-9a4c-4e57-af7f-c03e44c37ef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:42.278824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280586a3-9a4c-4e57-af7f-c03e44c37ef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:42.280465 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280586a3-9a4c-4e57-af7f-c03e44c37ef7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:42.291330 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1397/2774] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:17:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:17:42.295412 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a7be8-27f7-4e7e-8f9a-378b46242739 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:42.305945 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a7be8-27f7-4e7e-8f9a-378b46242739 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:42.310428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8a0c4-0be2-461c-8951-7e0628348648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:42.311067 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8a0c4-0be2-461c-8951-7e0628348648 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:42.311346 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8a0c4-0be2-461c-8951-7e0628348648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:42.311616 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8a0c4-0be2-461c-8951-7e0628348648 admin admin] PATH_INFO: `/v3/projects/8a6a618b352e47b99f4e2dfa51066215/users/a176f13d7e1f4aa383f8c4f6a8866849/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:42.325900 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f8a0c4-0be2-461c-8951-7e0628348648 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a6a618b352e47b99f4e2dfa51066215, user_id=a176f13d7e1f4aa383f8c4f6a8866849, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:42.328588 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f8a0c4-0be2-461c-8951-7e0628348648 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:42.405747 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1378/2775] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:17:42 2022] PUT /identity/v3/projects/8a6a618b352e47b99f4e2dfa51066215/users/a176f13d7e1f4aa383f8c4f6a8866849/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:17:42.409797 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280586a3-9a4c-4e57-af7f-c03e44c37ef7 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:42.420977 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280586a3-9a4c-4e57-af7f-c03e44c37ef7 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:42.479116 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9674dca-2395-4e00-a416-1d1009217f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:42.479765 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9674dca-2395-4e00-a416-1d1009217f2e admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:42.480039 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9674dca-2395-4e00-a416-1d1009217f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:42.480335 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9674dca-2395-4e00-a416-1d1009217f2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:42.480971 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9674dca-2395-4e00-a416-1d1009217f2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:42.482540 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9674dca-2395-4e00-a416-1d1009217f2e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:42.492446 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1398/2776] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:17:42 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:17:42.496773 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8a0c4-0be2-461c-8951-7e0628348648 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:42.507355 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8a0c4-0be2-461c-8951-7e0628348648 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:42.512001 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316d9e9-b2e1-4de3-9fa2-b59d181e7f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:42.513107 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316d9e9-b2e1-4de3-9fa2-b59d181e7f6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:42.513383 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316d9e9-b2e1-4de3-9fa2-b59d181e7f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:42.513653 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316d9e9-b2e1-4de3-9fa2-b59d181e7f6f admin admin] PATH_INFO: `/v3/projects/8a6a618b352e47b99f4e2dfa51066215/users/a176f13d7e1f4aa383f8c4f6a8866849/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:42.515548 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316d9e9-b2e1-4de3-9fa2-b59d181e7f6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a6a618b352e47b99f4e2dfa51066215, user_id=a176f13d7e1f4aa383f8c4f6a8866849, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:42.518236 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316d9e9-b2e1-4de3-9fa2-b59d181e7f6f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:42.529161 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1379/2777] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:17:42 2022] PUT /identity/v3/projects/8a6a618b352e47b99f4e2dfa51066215/users/a176f13d7e1f4aa383f8c4f6a8866849/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:17:42.539683 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710c553-859e-44ca-a70f-dede547e7245 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:42.539976 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710c553-859e-44ca-a70f-dede547e7245 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:42.540301 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710c553-859e-44ca-a70f-dede547e7245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:42.570949 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c710c553-859e-44ca-a70f-dede547e7245 None None] MFA Rules not processed for user `a176f13d7e1f4aa383f8c4f6a8866849`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:42.572220 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c710c553-859e-44ca-a70f-dede547e7245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:42.639377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1399/2778] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:17:42 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:17:42.651290 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316d9e9-b2e1-4de3-9fa2-b59d181e7f6f admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:42.653022 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b316d9e9-b2e1-4de3-9fa2-b59d181e7f6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:42.709551 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316d9e9-b2e1-4de3-9fa2-b59d181e7f6f admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:42.713163 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe37d9b-99fe-4ed2-9977-e4e74db570a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:42.713754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe37d9b-99fe-4ed2-9977-e4e74db570a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:42.714033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe37d9b-99fe-4ed2-9977-e4e74db570a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:42.714305 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe37d9b-99fe-4ed2-9977-e4e74db570a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:42.731669 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:42.771112 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe37d9b-99fe-4ed2-9977-e4e74db570a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:42.772526 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe37d9b-99fe-4ed2-9977-e4e74db570a1 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:42.781303 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1380/2779] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:17:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:17:44.680890 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710c553-859e-44ca-a70f-dede547e7245 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:44.682290 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c710c553-859e-44ca-a70f-dede547e7245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:44.694881 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710c553-859e-44ca-a70f-dede547e7245 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:44.698487 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9fe5e-d85d-4cbb-806e-07271e667fc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:44.699077 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9fe5e-d85d-4cbb-806e-07271e667fc2 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:44.699355 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9fe5e-d85d-4cbb-806e-07271e667fc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:44.699624 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9fe5e-d85d-4cbb-806e-07271e667fc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:44.747977 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:44.756750 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd9fe5e-d85d-4cbb-806e-07271e667fc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:44.758253 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd9fe5e-d85d-4cbb-806e-07271e667fc2 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:44.766840 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1400/2780] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:17:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:17:46.070980 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe37d9b-99fe-4ed2-9977-e4e74db570a1 service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:46.079446 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe37d9b-99fe-4ed2-9977-e4e74db570a1 service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:46.083281 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f180c-a67f-4df1-94cd-84e90e286a04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:46.083870 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f180c-a67f-4df1-94cd-84e90e286a04 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:46.084186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f180c-a67f-4df1-94cd-84e90e286a04 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:46.084459 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f180c-a67f-4df1-94cd-84e90e286a04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:46.093189 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952f180c-a67f-4df1-94cd-84e90e286a04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:46.094600 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952f180c-a67f-4df1-94cd-84e90e286a04 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:46.104634 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1381/2781] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:17:46 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:17:51.810240 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9fe5e-d85d-4cbb-806e-07271e667fc2 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:51.872402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9fe5e-d85d-4cbb-806e-07271e667fc2 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:51.876798 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e76391-38f0-447e-a25b-bc77f9ea875f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:51.877416 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e76391-38f0-447e-a25b-bc77f9ea875f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:51.877693 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e76391-38f0-447e-a25b-bc77f9ea875f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:51.877968 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e76391-38f0-447e-a25b-bc77f9ea875f admin admin] PATH_INFO: `/v3/users/ab66b0f97a0e4b49a59377fa4dac5111` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:51.879078 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e76391-38f0-447e-a25b-bc77f9ea875f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab66b0f97a0e4b49a59377fa4dac5111)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:51.881037 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e76391-38f0-447e-a25b-bc77f9ea875f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:52.027343 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a0e76391-38f0-447e-a25b-bc77f9ea875f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab66b0f97a0e4b49a59377fa4dac5111'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:52.027997 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a0e76391-38f0-447e-a25b-bc77f9ea875f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab66b0f97a0e4b49a59377fa4dac5111'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:52.028624 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a0e76391-38f0-447e-a25b-bc77f9ea875f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab66b0f97a0e4b49a59377fa4dac5111'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:52.036084 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a0e76391-38f0-447e-a25b-bc77f9ea875f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab66b0f97a0e4b49a59377fa4dac5111'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:52.047974 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a0e76391-38f0-447e-a25b-bc77f9ea875f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab66b0f97a0e4b49a59377fa4dac5111'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:52.119988 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1401/2782] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:17:51 2022] DELETE /identity/v3/users/ab66b0f97a0e4b49a59377fa4dac5111 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:17:52.124702 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db472d72-ce9a-4801-871d-6ca4248655c0 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:52.125169 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db472d72-ce9a-4801-871d-6ca4248655c0 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:52.125286 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db472d72-ce9a-4801-871d-6ca4248655c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:52.151055 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-db472d72-ce9a-4801-871d-6ca4248655c0 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:52.152331 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-db472d72-ce9a-4801-871d-6ca4248655c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:52.209752 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1382/2783] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:17:52.219486 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e76391-38f0-447e-a25b-bc77f9ea875f admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:52.220906 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a0e76391-38f0-447e-a25b-bc77f9ea875f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:52.277339 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e76391-38f0-447e-a25b-bc77f9ea875f admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:52.280944 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb386068-1276-47ce-90d1-5d6edf0059ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:52.281545 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb386068-1276-47ce-90d1-5d6edf0059ec service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:52.281823 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb386068-1276-47ce-90d1-5d6edf0059ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:52.282094 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb386068-1276-47ce-90d1-5d6edf0059ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:52.291654 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb386068-1276-47ce-90d1-5d6edf0059ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:52.293116 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb386068-1276-47ce-90d1-5d6edf0059ec service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:52.302342 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1402/2784] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:17:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:17:52.679749 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db472d72-ce9a-4801-871d-6ca4248655c0 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:52.681204 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-db472d72-ce9a-4801-871d-6ca4248655c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:52.693918 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db472d72-ce9a-4801-871d-6ca4248655c0 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:52.698549 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc0dc6-87d2-43d2-85b2-8924599931af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:52.699423 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc0dc6-87d2-43d2-85b2-8924599931af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:52.699423 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc0dc6-87d2-43d2-85b2-8924599931af admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:52.699712 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc0dc6-87d2-43d2-85b2-8924599931af admin admin] PATH_INFO: `/v3/projects/772b42b45d054104ab93ceb8a2cc4a9d` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:52.700941 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cc0dc6-87d2-43d2-85b2-8924599931af admin admin] RBAC: Authorizing `identity:delete_project(project_id=772b42b45d054104ab93ceb8a2cc4a9d)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:52.702904 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cc0dc6-87d2-43d2-85b2-8924599931af admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:52.810835 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-20cc0dc6-87d2-43d2-85b2-8924599931af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '772b42b45d054104ab93ceb8a2cc4a9d', 'request_id': 'req-20cc0dc6-87d2-43d2-85b2-8924599931af'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:52.885436 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-20cc0dc6-87d2-43d2-85b2-8924599931af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '772b42b45d054104ab93ceb8a2cc4a9d', 'request_id': 'req-20cc0dc6-87d2-43d2-85b2-8924599931af'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:52.891885 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-20cc0dc6-87d2-43d2-85b2-8924599931af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '772b42b45d054104ab93ceb8a2cc4a9d', 'request_id': 'req-20cc0dc6-87d2-43d2-85b2-8924599931af'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:52.902931 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-20cc0dc6-87d2-43d2-85b2-8924599931af admin admin] The token cache is being invalidate because project 772b42b45d054104ab93ceb8a2cc4a9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:17:52.903219 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-20cc0dc6-87d2-43d2-85b2-8924599931af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:52.903841 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-20cc0dc6-87d2-43d2-85b2-8924599931af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:52.905338 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1383/2785] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:17:52 2022] DELETE /identity/v3/projects/772b42b45d054104ab93ceb8a2cc4a9d => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:17:52.909367 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb386068-1276-47ce-90d1-5d6edf0059ec service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:52.919285 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb386068-1276-47ce-90d1-5d6edf0059ec service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:52.929468 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:52.974234 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f307b72-db63-4524-8f9f-bda40200976c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:52.974884 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f307b72-db63-4524-8f9f-bda40200976c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:52.975162 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f307b72-db63-4524-8f9f-bda40200976c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:52.975431 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f307b72-db63-4524-8f9f-bda40200976c admin admin] PATH_INFO: `/v3/users/74b2855f803949959d1fce08f6a48880` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:52.976670 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f307b72-db63-4524-8f9f-bda40200976c admin admin] RBAC: Authorizing `identity:delete_user(user_id=74b2855f803949959d1fce08f6a48880)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:52.980387 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f307b72-db63-4524-8f9f-bda40200976c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:53.204229 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7f307b72-db63-4524-8f9f-bda40200976c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74b2855f803949959d1fce08f6a48880'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:53.204843 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7f307b72-db63-4524-8f9f-bda40200976c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74b2855f803949959d1fce08f6a48880'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:53.205444 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7f307b72-db63-4524-8f9f-bda40200976c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74b2855f803949959d1fce08f6a48880'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:53.213888 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7f307b72-db63-4524-8f9f-bda40200976c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74b2855f803949959d1fce08f6a48880'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:53.226508 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7f307b72-db63-4524-8f9f-bda40200976c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74b2855f803949959d1fce08f6a48880'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:53.289899 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1403/2786] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:17:52 2022] DELETE /identity/v3/users/74b2855f803949959d1fce08f6a48880 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:17:53.741790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc0dc6-87d2-43d2-85b2-8924599931af admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:53.741790 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-20cc0dc6-87d2-43d2-85b2-8924599931af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:53.751680 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc0dc6-87d2-43d2-85b2-8924599931af admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:53.780478 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:53.811023 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f612a-0b8c-4f8f-9d45-25c2040a78bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:53.811660 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f612a-0b8c-4f8f-9d45-25c2040a78bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:53.811941 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f612a-0b8c-4f8f-9d45-25c2040a78bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:53.812253 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f612a-0b8c-4f8f-9d45-25c2040a78bc admin admin] PATH_INFO: `/v3/projects/8219f46e43e7468ba707ce9a60ac3875` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:53.813363 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f612a-0b8c-4f8f-9d45-25c2040a78bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8219f46e43e7468ba707ce9a60ac3875)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:53.815268 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f612a-0b8c-4f8f-9d45-25c2040a78bc admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:53.930277 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-da0f612a-0b8c-4f8f-9d45-25c2040a78bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8219f46e43e7468ba707ce9a60ac3875', 'request_id': 'req-da0f612a-0b8c-4f8f-9d45-25c2040a78bc'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:54.000343 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-da0f612a-0b8c-4f8f-9d45-25c2040a78bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8219f46e43e7468ba707ce9a60ac3875', 'request_id': 'req-da0f612a-0b8c-4f8f-9d45-25c2040a78bc'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:54.007213 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-da0f612a-0b8c-4f8f-9d45-25c2040a78bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8219f46e43e7468ba707ce9a60ac3875', 'request_id': 'req-da0f612a-0b8c-4f8f-9d45-25c2040a78bc'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:54.017734 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-da0f612a-0b8c-4f8f-9d45-25c2040a78bc admin admin] The token cache is being invalidate because project 8219f46e43e7468ba707ce9a60ac3875 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:17:54.018025 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-da0f612a-0b8c-4f8f-9d45-25c2040a78bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:54.018673 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-da0f612a-0b8c-4f8f-9d45-25c2040a78bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:54.020694 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1384/2787] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:17:53 2022] DELETE /identity/v3/projects/8219f46e43e7468ba707ce9a60ac3875 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:17:54.024595 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f307b72-db63-4524-8f9f-bda40200976c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:54.026005 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7f307b72-db63-4524-8f9f-bda40200976c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:54.039169 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f307b72-db63-4524-8f9f-bda40200976c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:54.095051 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372265d-736d-48dc-a380-2138cf2afeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:54.095683 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372265d-736d-48dc-a380-2138cf2afeaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:54.095967 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372265d-736d-48dc-a380-2138cf2afeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:54.096272 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372265d-736d-48dc-a380-2138cf2afeaf admin admin] PATH_INFO: `/v3/users/a176f13d7e1f4aa383f8c4f6a8866849` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:54.097392 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8372265d-736d-48dc-a380-2138cf2afeaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a176f13d7e1f4aa383f8c4f6a8866849)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:54.099378 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8372265d-736d-48dc-a380-2138cf2afeaf admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:54.267016 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:54.276700 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8372265d-736d-48dc-a380-2138cf2afeaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a176f13d7e1f4aa383f8c4f6a8866849'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:54.277359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8372265d-736d-48dc-a380-2138cf2afeaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a176f13d7e1f4aa383f8c4f6a8866849'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:54.277966 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8372265d-736d-48dc-a380-2138cf2afeaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a176f13d7e1f4aa383f8c4f6a8866849'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:54.284856 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8372265d-736d-48dc-a380-2138cf2afeaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a176f13d7e1f4aa383f8c4f6a8866849'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:54.296513 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8372265d-736d-48dc-a380-2138cf2afeaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a176f13d7e1f4aa383f8c4f6a8866849'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:54.385917 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1404/2788] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:17:54 2022] DELETE /identity/v3/users/a176f13d7e1f4aa383f8c4f6a8866849 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:17:54.800905 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f612a-0b8c-4f8f-9d45-25c2040a78bc admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:54.802740 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-da0f612a-0b8c-4f8f-9d45-25c2040a78bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:54.815589 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f612a-0b8c-4f8f-9d45-25c2040a78bc admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:54.872371 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c483883-5f15-4502-9d48-cfb25d82f1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:54.873044 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c483883-5f15-4502-9d48-cfb25d82f1e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:54.873332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c483883-5f15-4502-9d48-cfb25d82f1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:54.873610 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c483883-5f15-4502-9d48-cfb25d82f1e1 admin admin] PATH_INFO: `/v3/projects/8a6a618b352e47b99f4e2dfa51066215` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:54.874728 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c483883-5f15-4502-9d48-cfb25d82f1e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a6a618b352e47b99f4e2dfa51066215)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:54.876676 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c483883-5f15-4502-9d48-cfb25d82f1e1 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:54.975415 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0c483883-5f15-4502-9d48-cfb25d82f1e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a6a618b352e47b99f4e2dfa51066215', 'request_id': 'req-0c483883-5f15-4502-9d48-cfb25d82f1e1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:55.043649 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0c483883-5f15-4502-9d48-cfb25d82f1e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a6a618b352e47b99f4e2dfa51066215', 'request_id': 'req-0c483883-5f15-4502-9d48-cfb25d82f1e1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:55.050507 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0c483883-5f15-4502-9d48-cfb25d82f1e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a6a618b352e47b99f4e2dfa51066215', 'request_id': 'req-0c483883-5f15-4502-9d48-cfb25d82f1e1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:55.051739 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:55.061304 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0c483883-5f15-4502-9d48-cfb25d82f1e1 admin admin] The token cache is being invalidate because project 8a6a618b352e47b99f4e2dfa51066215 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:17:55.061593 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0c483883-5f15-4502-9d48-cfb25d82f1e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:55.062228 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0c483883-5f15-4502-9d48-cfb25d82f1e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:55.063697 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1385/2789] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:17:54 2022] DELETE /identity/v3/projects/8a6a618b352e47b99f4e2dfa51066215 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:17:55.069891 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1b01c-441f-4c88-894d-1c6a13152920 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:55.070392 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1b01c-441f-4c88-894d-1c6a13152920 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:55.070392 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1b01c-441f-4c88-894d-1c6a13152920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:55.090674 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-93d1b01c-441f-4c88-894d-1c6a13152920 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:55.091876 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-93d1b01c-441f-4c88-894d-1c6a13152920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:55.149752 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1405/2790] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:17:55.158921 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47005755-5cad-49ed-bd09-039492527fb9 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:55.159251 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47005755-5cad-49ed-bd09-039492527fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:55.159530 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47005755-5cad-49ed-bd09-039492527fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:55.179577 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-47005755-5cad-49ed-bd09-039492527fb9 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:55.181528 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-47005755-5cad-49ed-bd09-039492527fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:55.189448 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1386/2791] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:17:55.194150 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1b01c-441f-4c88-894d-1c6a13152920 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:55.203606 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1b01c-441f-4c88-894d-1c6a13152920 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:55.208386 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b4917a-a896-440b-9587-841f19d40fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:55.208969 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4917a-a896-440b-9587-841f19d40fca admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:55.209244 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4917a-a896-440b-9587-841f19d40fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:55.209513 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4917a-a896-440b-9587-841f19d40fca admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:55.210149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b4917a-a896-440b-9587-841f19d40fca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:17:55.210447 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b4917a-a896-440b-9587-841f19d40fca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:55.212030 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b4917a-a896-440b-9587-841f19d40fca admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:55.223315 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1406/2792] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:17:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:17:55.227138 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47005755-5cad-49ed-bd09-039492527fb9 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:55.236193 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47005755-5cad-49ed-bd09-039492527fb9 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:55.241370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a5977-c34a-4b2d-9de0-9496ae9b0fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:55.241948 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a5977-c34a-4b2d-9de0-9496ae9b0fb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:55.242226 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a5977-c34a-4b2d-9de0-9496ae9b0fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:55.242494 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a5977-c34a-4b2d-9de0-9496ae9b0fb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:55.243116 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a5977-c34a-4b2d-9de0-9496ae9b0fb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:55.245044 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a5977-c34a-4b2d-9de0-9496ae9b0fb6 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:55.320734 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1387/2793] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:17:55 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:17:55.324741 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b4917a-a896-440b-9587-841f19d40fca admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:55.334734 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b4917a-a896-440b-9587-841f19d40fca admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:55.389616 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a150078-b8c8-48f2-8944-31787d84a6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:55.390236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a150078-b8c8-48f2-8944-31787d84a6bf admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:55.390512 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a150078-b8c8-48f2-8944-31787d84a6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:55.390780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a150078-b8c8-48f2-8944-31787d84a6bf admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:55.391408 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a150078-b8c8-48f2-8944-31787d84a6bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:55.393337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a150078-b8c8-48f2-8944-31787d84a6bf admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:55.462878 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1407/2794] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:17:55 2022] POST /identity/v3/users => generated 292 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:17:55.466951 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a5977-c34a-4b2d-9de0-9496ae9b0fb6 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:55.477717 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a5977-c34a-4b2d-9de0-9496ae9b0fb6 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:55.482473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2257a819-88f1-4e54-be81-660ef64f0fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:55.483054 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257a819-88f1-4e54-be81-660ef64f0fda admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:55.483323 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257a819-88f1-4e54-be81-660ef64f0fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:55.483594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257a819-88f1-4e54-be81-660ef64f0fda admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:55.484384 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2257a819-88f1-4e54-be81-660ef64f0fda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:55.486074 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2257a819-88f1-4e54-be81-660ef64f0fda admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:55.496802 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1388/2795] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:17:55.500768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a150078-b8c8-48f2-8944-31787d84a6bf admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:55.511255 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a150078-b8c8-48f2-8944-31787d84a6bf admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:55.515880 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78981b2-95f0-4865-a745-ea5dd2c20541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:55.516506 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78981b2-95f0-4865-a745-ea5dd2c20541 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:55.516787 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78981b2-95f0-4865-a745-ea5dd2c20541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:55.517062 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78981b2-95f0-4865-a745-ea5dd2c20541 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:55.517700 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78981b2-95f0-4865-a745-ea5dd2c20541 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:55.519377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78981b2-95f0-4865-a745-ea5dd2c20541 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:55.529861 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1408/2796] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:17:55.533818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2257a819-88f1-4e54-be81-660ef64f0fda admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:55.544275 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2257a819-88f1-4e54-be81-660ef64f0fda admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:55.548867 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dde434-286f-4caf-8442-22180835d9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:55.549505 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dde434-286f-4caf-8442-22180835d9a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:55.549781 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dde434-286f-4caf-8442-22180835d9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:55.550053 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dde434-286f-4caf-8442-22180835d9a5 admin admin] PATH_INFO: `/v3/projects/c4c0ef9a4c9b42b49e7a478910516780/users/c5d209fe7fcf451699c3199691711fd3/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:55.565011 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dde434-286f-4caf-8442-22180835d9a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4c0ef9a4c9b42b49e7a478910516780, user_id=c5d209fe7fcf451699c3199691711fd3, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:55.567659 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dde434-286f-4caf-8442-22180835d9a5 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:55.639169 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1389/2797] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:17:55 2022] PUT /identity/v3/projects/c4c0ef9a4c9b42b49e7a478910516780/users/c5d209fe7fcf451699c3199691711fd3/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:17:55.649329 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca513b-e083-4df9-9e13-b93ffb66b791 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:55.649544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca513b-e083-4df9-9e13-b93ffb66b791 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:55.649817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca513b-e083-4df9-9e13-b93ffb66b791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:55.678139 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-cdca513b-e083-4df9-9e13-b93ffb66b791 None None] MFA Rules not processed for user `c5d209fe7fcf451699c3199691711fd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:55.679334 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cdca513b-e083-4df9-9e13-b93ffb66b791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:55.684317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:55.743146 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1409/2798] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:17:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:17:55.754365 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dde434-286f-4caf-8442-22180835d9a5 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:55.756061 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-82dde434-286f-4caf-8442-22180835d9a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:55.812381 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dde434-286f-4caf-8442-22180835d9a5 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:55.815992 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a98899-4dcd-4222-ae08-6dff0b0afa24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:55.816620 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a98899-4dcd-4222-ae08-6dff0b0afa24 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:55.816901 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a98899-4dcd-4222-ae08-6dff0b0afa24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:55.817174 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a98899-4dcd-4222-ae08-6dff0b0afa24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:55.825415 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a98899-4dcd-4222-ae08-6dff0b0afa24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:55.826842 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a98899-4dcd-4222-ae08-6dff0b0afa24 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:55.835727 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1390/2799] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:17:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:17:58.845361 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca513b-e083-4df9-9e13-b93ffb66b791 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:58.907074 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca513b-e083-4df9-9e13-b93ffb66b791 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:58.912026 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b8fab-110e-47e3-b6d6-edf5e91920c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:58.912675 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b8fab-110e-47e3-b6d6-edf5e91920c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:58.912976 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b8fab-110e-47e3-b6d6-edf5e91920c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:58.913249 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b8fab-110e-47e3-b6d6-edf5e91920c9 admin admin] PATH_INFO: `/v3/users/c5d209fe7fcf451699c3199691711fd3` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:58.914357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803b8fab-110e-47e3-b6d6-edf5e91920c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5d209fe7fcf451699c3199691711fd3)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:58.916293 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803b8fab-110e-47e3-b6d6-edf5e91920c9 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:59.068698 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-803b8fab-110e-47e3-b6d6-edf5e91920c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5d209fe7fcf451699c3199691711fd3'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:59.069353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-803b8fab-110e-47e3-b6d6-edf5e91920c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5d209fe7fcf451699c3199691711fd3'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:59.069951 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-803b8fab-110e-47e3-b6d6-edf5e91920c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5d209fe7fcf451699c3199691711fd3'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:59.077064 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-803b8fab-110e-47e3-b6d6-edf5e91920c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5d209fe7fcf451699c3199691711fd3'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:59.089683 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-803b8fab-110e-47e3-b6d6-edf5e91920c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5d209fe7fcf451699c3199691711fd3'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:59.153033 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1410/2800] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:17:58 2022] DELETE /identity/v3/users/c5d209fe7fcf451699c3199691711fd3 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:17:59.157515 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9fcc2-542f-457d-a7d7-e7841661ed10 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:59.157982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9fcc2-542f-457d-a7d7-e7841661ed10 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:59.158111 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9fcc2-542f-457d-a7d7-e7841661ed10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:59.177483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-efe9fcc2-542f-457d-a7d7-e7841661ed10 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:59.178669 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-efe9fcc2-542f-457d-a7d7-e7841661ed10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:59.236150 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1391/2801] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:17:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:17:59.245487 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b8fab-110e-47e3-b6d6-edf5e91920c9 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:59.246908 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-803b8fab-110e-47e3-b6d6-edf5e91920c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:59.277373 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:59.302661 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b8fab-110e-47e3-b6d6-edf5e91920c9 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:59.306296 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55701e-908a-46c5-ad02-cd0dfc710be8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:59.306889 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55701e-908a-46c5-ad02-cd0dfc710be8 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:59.307165 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55701e-908a-46c5-ad02-cd0dfc710be8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:59.307438 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55701e-908a-46c5-ad02-cd0dfc710be8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:59.317300 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55701e-908a-46c5-ad02-cd0dfc710be8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:59.318752 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55701e-908a-46c5-ad02-cd0dfc710be8 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:59.327861 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1411/2802] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:17:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:17:59.724800 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe9fcc2-542f-457d-a7d7-e7841661ed10 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:59.726254 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-efe9fcc2-542f-457d-a7d7-e7841661ed10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:59.738983 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe9fcc2-542f-457d-a7d7-e7841661ed10 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:59.743689 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862082f7-88fd-4af5-a20d-876212c29efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:59.744352 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862082f7-88fd-4af5-a20d-876212c29efb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:59.744630 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862082f7-88fd-4af5-a20d-876212c29efb admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:59.744902 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862082f7-88fd-4af5-a20d-876212c29efb admin admin] PATH_INFO: `/v3/projects/c4c0ef9a4c9b42b49e7a478910516780` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:59.746046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862082f7-88fd-4af5-a20d-876212c29efb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4c0ef9a4c9b42b49e7a478910516780)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:59.747941 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862082f7-88fd-4af5-a20d-876212c29efb admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:59.773406 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:59.871774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-862082f7-88fd-4af5-a20d-876212c29efb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4c0ef9a4c9b42b49e7a478910516780', 'request_id': 'req-862082f7-88fd-4af5-a20d-876212c29efb'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:59.934613 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-862082f7-88fd-4af5-a20d-876212c29efb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4c0ef9a4c9b42b49e7a478910516780', 'request_id': 'req-862082f7-88fd-4af5-a20d-876212c29efb'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:59.941028 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-862082f7-88fd-4af5-a20d-876212c29efb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4c0ef9a4c9b42b49e7a478910516780', 'request_id': 'req-862082f7-88fd-4af5-a20d-876212c29efb'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:59.951721 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-862082f7-88fd-4af5-a20d-876212c29efb admin admin] The token cache is being invalidate because project c4c0ef9a4c9b42b49e7a478910516780 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:17:59.952088 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-862082f7-88fd-4af5-a20d-876212c29efb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:59.952718 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-862082f7-88fd-4af5-a20d-876212c29efb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:17:59.954219 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1392/2803] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:17:59 2022] DELETE /identity/v3/projects/c4c0ef9a4c9b42b49e7a478910516780 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:17:59.960491 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076031f-90a6-4fe4-9e84-9f6eb19597a9 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:59.964564 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076031f-90a6-4fe4-9e84-9f6eb19597a9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:59.964564 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076031f-90a6-4fe4-9e84-9f6eb19597a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:59.982597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c076031f-90a6-4fe4-9e84-9f6eb19597a9 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:59.983876 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c076031f-90a6-4fe4-9e84-9f6eb19597a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:00.053670 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1412/2804] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:17:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:18:00.061756 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b75b27-7289-46bb-b95a-e8566b2d311d None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:00.062043 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b75b27-7289-46bb-b95a-e8566b2d311d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:00.062322 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b75b27-7289-46bb-b95a-e8566b2d311d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:00.082275 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-87b75b27-7289-46bb-b95a-e8566b2d311d None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:18:00.083470 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-87b75b27-7289-46bb-b95a-e8566b2d311d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:00.091396 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1393/2805] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:18:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:18:00.095589 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076031f-90a6-4fe4-9e84-9f6eb19597a9 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:00.105911 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076031f-90a6-4fe4-9e84-9f6eb19597a9 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:00.110456 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288badad-9afb-4697-b4e2-2b7907fbc911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:00.111042 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288badad-9afb-4697-b4e2-2b7907fbc911 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:00.111346 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288badad-9afb-4697-b4e2-2b7907fbc911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:00.111615 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288badad-9afb-4697-b4e2-2b7907fbc911 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:00.112283 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288badad-9afb-4697-b4e2-2b7907fbc911 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:18:00.112588 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288badad-9afb-4697-b4e2-2b7907fbc911 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:00.114824 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288badad-9afb-4697-b4e2-2b7907fbc911 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:00.126142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1413/2806] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:18:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:18:00.130437 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b75b27-7289-46bb-b95a-e8566b2d311d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:00.139637 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b75b27-7289-46bb-b95a-e8566b2d311d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:00.144337 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ca878-c5cb-40b7-816a-22bb6a714f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:00.144916 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ca878-c5cb-40b7-816a-22bb6a714f83 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:00.149250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ca878-c5cb-40b7-816a-22bb6a714f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:00.149250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ca878-c5cb-40b7-816a-22bb6a714f83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:00.149250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ca878-c5cb-40b7-816a-22bb6a714f83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:00.149250 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ca878-c5cb-40b7-816a-22bb6a714f83 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:00.212755 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1394/2807] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:18:00 2022] POST /identity/v3/projects => generated 385 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:18:00.216758 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288badad-9afb-4697-b4e2-2b7907fbc911 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:00.227236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288badad-9afb-4697-b4e2-2b7907fbc911 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:00.281682 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f891de0d-b0c6-4cb9-b4ef-32976c976d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:00.282276 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891de0d-b0c6-4cb9-b4ef-32976c976d04 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:00.282553 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891de0d-b0c6-4cb9-b4ef-32976c976d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:00.282823 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891de0d-b0c6-4cb9-b4ef-32976c976d04 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:00.283452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f891de0d-b0c6-4cb9-b4ef-32976c976d04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:00.285979 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f891de0d-b0c6-4cb9-b4ef-32976c976d04 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:00.370712 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1414/2808] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:18:00 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:18:00.374787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ca878-c5cb-40b7-816a-22bb6a714f83 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:00.384998 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ca878-c5cb-40b7-816a-22bb6a714f83 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:00.389548 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b58bd5-06f1-4318-ac58-ee73b1bc2121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:00.390135 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b58bd5-06f1-4318-ac58-ee73b1bc2121 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:00.390415 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b58bd5-06f1-4318-ac58-ee73b1bc2121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:00.390685 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b58bd5-06f1-4318-ac58-ee73b1bc2121 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:00.391320 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b58bd5-06f1-4318-ac58-ee73b1bc2121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:00.392943 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b58bd5-06f1-4318-ac58-ee73b1bc2121 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:00.402618 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1395/2809] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:18:00 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:18:00.406497 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f891de0d-b0c6-4cb9-b4ef-32976c976d04 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:00.416874 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f891de0d-b0c6-4cb9-b4ef-32976c976d04 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:00.421469 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bdcbc5-8f20-4e9d-831b-ebdd92088601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:00.422056 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bdcbc5-8f20-4e9d-831b-ebdd92088601 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:00.422337 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bdcbc5-8f20-4e9d-831b-ebdd92088601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:00.422615 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bdcbc5-8f20-4e9d-831b-ebdd92088601 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:00.423254 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bdcbc5-8f20-4e9d-831b-ebdd92088601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:00.424880 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bdcbc5-8f20-4e9d-831b-ebdd92088601 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:00.435910 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1415/2810] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:18:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:18:00.439978 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b58bd5-06f1-4318-ac58-ee73b1bc2121 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:00.449877 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b58bd5-06f1-4318-ac58-ee73b1bc2121 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:00.454413 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb3fe0c-7050-4f97-b776-f6fbae7eb301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:00.455047 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3fe0c-7050-4f97-b776-f6fbae7eb301 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:00.455323 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3fe0c-7050-4f97-b776-f6fbae7eb301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:00.455594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3fe0c-7050-4f97-b776-f6fbae7eb301 admin admin] PATH_INFO: `/v3/projects/a1261e5e82414293b96ce77482498a89/users/fa04bcfffac2473baad2953f1c64eb95/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:00.471190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb3fe0c-7050-4f97-b776-f6fbae7eb301 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1261e5e82414293b96ce77482498a89, user_id=fa04bcfffac2473baad2953f1c64eb95, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:00.474482 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb3fe0c-7050-4f97-b776-f6fbae7eb301 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:00.537541 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1396/2811] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:18:00 2022] PUT /identity/v3/projects/a1261e5e82414293b96ce77482498a89/users/fa04bcfffac2473baad2953f1c64eb95/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:18:00.543064 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9043915-eaf7-4079-964e-dfa82cb30ce1 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:00.543351 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9043915-eaf7-4079-964e-dfa82cb30ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:00.543622 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9043915-eaf7-4079-964e-dfa82cb30ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:00.562410 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d9043915-eaf7-4079-964e-dfa82cb30ce1 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:18:00.563632 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d9043915-eaf7-4079-964e-dfa82cb30ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:00.593461 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:00.621763 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1416/2812] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:18:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:18:00.631479 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb3fe0c-7050-4f97-b776-f6fbae7eb301 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:00.633247 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-1cb3fe0c-7050-4f97-b776-f6fbae7eb301 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:00.688756 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb3fe0c-7050-4f97-b776-f6fbae7eb301 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:00.692449 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757dbba7-404f-4631-90dd-a29f69426b3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:00.693036 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757dbba7-404f-4631-90dd-a29f69426b3e service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:00.693309 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757dbba7-404f-4631-90dd-a29f69426b3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:00.693576 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757dbba7-404f-4631-90dd-a29f69426b3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:00.702280 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757dbba7-404f-4631-90dd-a29f69426b3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:00.703688 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757dbba7-404f-4631-90dd-a29f69426b3e service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:00.713671 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1397/2813] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:18:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:18:05.919373 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583c745-8070-409f-a361-56e69883f1b7 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:05.919373 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583c745-8070-409f-a361-56e69883f1b7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:05.919373 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583c745-8070-409f-a361-56e69883f1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:05.961452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e583c745-8070-409f-a361-56e69883f1b7 None None] MFA Rules not processed for user `fa04bcfffac2473baad2953f1c64eb95`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:18:05.962902 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e583c745-8070-409f-a361-56e69883f1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:06.032209 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1417/2814] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:18:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:18:06.037081 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757dbba7-404f-4631-90dd-a29f69426b3e service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:06.046867 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757dbba7-404f-4631-90dd-a29f69426b3e service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:06.051664 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf66aa6-b10f-4f83-bb40-4e9dc8f0f7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:06.052351 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf66aa6-b10f-4f83-bb40-4e9dc8f0f7f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:06.052631 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf66aa6-b10f-4f83-bb40-4e9dc8f0f7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:06.052904 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf66aa6-b10f-4f83-bb40-4e9dc8f0f7f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:06.053545 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf66aa6-b10f-4f83-bb40-4e9dc8f0f7f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:06.055456 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf66aa6-b10f-4f83-bb40-4e9dc8f0f7f3 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:06.142631 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1398/2815] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:18:06 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:18:06.147424 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e583c745-8070-409f-a361-56e69883f1b7 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:06.159116 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e583c745-8070-409f-a361-56e69883f1b7 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:06.225080 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaa75b4-ce3e-49af-84bc-aeb34b2da1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:06.226079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa75b4-ce3e-49af-84bc-aeb34b2da1cb admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:06.226567 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa75b4-ce3e-49af-84bc-aeb34b2da1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:06.227061 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa75b4-ce3e-49af-84bc-aeb34b2da1cb admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:06.228221 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaa75b4-ce3e-49af-84bc-aeb34b2da1cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:06.231357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaa75b4-ce3e-49af-84bc-aeb34b2da1cb admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:06.320934 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1418/2816] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:18:06 2022] POST /identity/v3/users => generated 298 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:18:06.325148 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf66aa6-b10f-4f83-bb40-4e9dc8f0f7f3 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:06.336302 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf66aa6-b10f-4f83-bb40-4e9dc8f0f7f3 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:06.340798 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2c912-5cf4-42ee-8cf7-63d3e96a18b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:06.341451 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2c912-5cf4-42ee-8cf7-63d3e96a18b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:06.341729 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2c912-5cf4-42ee-8cf7-63d3e96a18b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:06.341998 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2c912-5cf4-42ee-8cf7-63d3e96a18b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:06.342628 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2c912-5cf4-42ee-8cf7-63d3e96a18b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:06.344311 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2c912-5cf4-42ee-8cf7-63d3e96a18b7 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:06.354727 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1399/2817] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:18:06.358732 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaa75b4-ce3e-49af-84bc-aeb34b2da1cb admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:06.368712 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaa75b4-ce3e-49af-84bc-aeb34b2da1cb admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:06.373479 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a924ad-2163-43d0-bbdd-222df8999ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:06.374185 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a924ad-2163-43d0-bbdd-222df8999ffa admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:06.374463 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a924ad-2163-43d0-bbdd-222df8999ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:06.374759 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a924ad-2163-43d0-bbdd-222df8999ffa admin admin] PATH_INFO: `/v3/projects/ca8416bca7fb45a3a30c90a17dea4c86/users/ffd2599d3dcf4b708766cf963f73616d/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:06.389941 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a924ad-2163-43d0-bbdd-222df8999ffa admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca8416bca7fb45a3a30c90a17dea4c86, user_id=ffd2599d3dcf4b708766cf963f73616d, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:06.392609 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a924ad-2163-43d0-bbdd-222df8999ffa admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:06.497602 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1419/2818] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:18:06 2022] PUT /identity/v3/projects/ca8416bca7fb45a3a30c90a17dea4c86/users/ffd2599d3dcf4b708766cf963f73616d/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:18:06.501774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2c912-5cf4-42ee-8cf7-63d3e96a18b7 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:06.512616 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2c912-5cf4-42ee-8cf7-63d3e96a18b7 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:06.524952 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:06.578883 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf922d5a-2d79-4945-89e7-d925d84ecf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:06.579494 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf922d5a-2d79-4945-89e7-d925d84ecf0e admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:06.579773 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf922d5a-2d79-4945-89e7-d925d84ecf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:06.580047 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf922d5a-2d79-4945-89e7-d925d84ecf0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:06.580722 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf922d5a-2d79-4945-89e7-d925d84ecf0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:06.582576 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf922d5a-2d79-4945-89e7-d925d84ecf0e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:06.592604 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1400/2819] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:18:06.596735 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a924ad-2163-43d0-bbdd-222df8999ffa admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:06.606863 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a924ad-2163-43d0-bbdd-222df8999ffa admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:06.611475 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b9fd88-698b-41c2-b109-8cf993559d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:06.612158 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9fd88-698b-41c2-b109-8cf993559d4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:06.612436 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9fd88-698b-41c2-b109-8cf993559d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:06.613201 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9fd88-698b-41c2-b109-8cf993559d4e admin admin] PATH_INFO: `/v3/projects/ca8416bca7fb45a3a30c90a17dea4c86/users/ffd2599d3dcf4b708766cf963f73616d/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:06.615184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b9fd88-698b-41c2-b109-8cf993559d4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca8416bca7fb45a3a30c90a17dea4c86, user_id=ffd2599d3dcf4b708766cf963f73616d, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:06.617840 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b9fd88-698b-41c2-b109-8cf993559d4e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:06.632534 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1420/2820] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:18:06 2022] PUT /identity/v3/projects/ca8416bca7fb45a3a30c90a17dea4c86/users/ffd2599d3dcf4b708766cf963f73616d/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:18:11.945457 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bdeb20-8c21-492e-b72b-45e54b7036e8 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:11.946035 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bdeb20-8c21-492e-b72b-45e54b7036e8 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:11.946035 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bdeb20-8c21-492e-b72b-45e54b7036e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:11.985234 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-b3bdeb20-8c21-492e-b72b-45e54b7036e8 None None] MFA Rules not processed for user `ffd2599d3dcf4b708766cf963f73616d`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:18:11.987047 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b3bdeb20-8c21-492e-b72b-45e54b7036e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:12.063685 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1401/2821] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:18:11 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:18:12.076347 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b9fd88-698b-41c2-b109-8cf993559d4e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:12.078084 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-16b9fd88-698b-41c2-b109-8cf993559d4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:12.100552 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:12.139357 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b9fd88-698b-41c2-b109-8cf993559d4e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:12.143071 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff18c51-08ff-4377-b445-0bcdff9fa096 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7832e97d7e97439e821268d05816ae3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:12.143798 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff18c51-08ff-4377-b445-0bcdff9fa096 service glance] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:12.144172 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff18c51-08ff-4377-b445-0bcdff9fa096 service glance] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:12.144445 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff18c51-08ff-4377-b445-0bcdff9fa096 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:12.204600 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff18c51-08ff-4377-b445-0bcdff9fa096 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:12.206115 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff18c51-08ff-4377-b445-0bcdff9fa096 service glance] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:12.216224 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1421/2822] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:18:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:18:12.450287 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23df58-5a8d-4454-8ae8-a4f2b786f3e0 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:12.450576 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23df58-5a8d-4454-8ae8-a4f2b786f3e0 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:12.450846 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23df58-5a8d-4454-8ae8-a4f2b786f3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:12.470839 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-ee23df58-5a8d-4454-8ae8-a4f2b786f3e0 None None] MFA Rules not processed for user `8c21ea7d3cbb494886e60a4bd4f39145`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:18:12.472779 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ee23df58-5a8d-4454-8ae8-a4f2b786f3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:12.528780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1402/2823] 10.222.0.48 () {58 vars in 1009 bytes} [Fri Sep 9 21:18:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:18:12.671041 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff18c51-08ff-4377-b445-0bcdff9fa096 service glance] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:12.672486 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9ff18c51-08ff-4377-b445-0bcdff9fa096 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:12.741026 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff18c51-08ff-4377-b445-0bcdff9fa096 service glance] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:12.746099 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99557b58-6a2b-4f2d-87cb-c6c191ec92cd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d28fa5d49d44120bd5eaec2d4e5fdbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:12.746706 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99557b58-6a2b-4f2d-87cb-c6c191ec92cd service swift] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:12.746994 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99557b58-6a2b-4f2d-87cb-c6c191ec92cd service swift] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:12.747269 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99557b58-6a2b-4f2d-87cb-c6c191ec92cd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:12.755767 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99557b58-6a2b-4f2d-87cb-c6c191ec92cd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:12.757414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99557b58-6a2b-4f2d-87cb-c6c191ec92cd service swift] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:12.766236 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1422/2824] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Sep 9 21:18:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 21:18:22.521480 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=80 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:22.556596 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab7cb2-1ead-430e-9e54-685862f19b2a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:22.556878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab7cb2-1ead-430e-9e54-685862f19b2a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:22.557176 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab7cb2-1ead-430e-9e54-685862f19b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:22.576383 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-aaab7cb2-1ead-430e-9e54-685862f19b2a None None] MFA Rules not processed for user `fa04bcfffac2473baad2953f1c64eb95`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:18:22.577575 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-aaab7cb2-1ead-430e-9e54-685862f19b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:22.585301 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1403/2825] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:18:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:18:22.762301 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=72 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:22.840841 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99557b58-6a2b-4f2d-87cb-c6c191ec92cd service swift] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:22.842246 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-99557b58-6a2b-4f2d-87cb-c6c191ec92cd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:22.898820 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99557b58-6a2b-4f2d-87cb-c6c191ec92cd service swift] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:22.902404 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196d43e-e17b-40e6-b963-c5d23d7a6173 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:22.903001 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196d43e-e17b-40e6-b963-c5d23d7a6173 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:22.903281 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196d43e-e17b-40e6-b963-c5d23d7a6173 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:22.903555 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196d43e-e17b-40e6-b963-c5d23d7a6173 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:22.905284 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f196d43e-e17b-40e6-b963-c5d23d7a6173 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:22.914531 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196d43e-e17b-40e6-b963-c5d23d7a6173 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:22.915954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196d43e-e17b-40e6-b963-c5d23d7a6173 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:22.923858 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1423/2826] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:18:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:18:23.309190 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab7cb2-1ead-430e-9e54-685862f19b2a None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:23.310615 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-aaab7cb2-1ead-430e-9e54-685862f19b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:23.386645 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab7cb2-1ead-430e-9e54-685862f19b2a None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:23.391606 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d13ee-5868-4b18-981b-17b58fdb9dc9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4f6de2065d4fbea3fa141da760f708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:23.392242 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d13ee-5868-4b18-981b-17b58fdb9dc9 service placement] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:23.392519 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d13ee-5868-4b18-981b-17b58fdb9dc9 service placement] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:23.392783 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d13ee-5868-4b18-981b-17b58fdb9dc9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:23.394494 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b20d13ee-5868-4b18-981b-17b58fdb9dc9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:23.405613 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d13ee-5868-4b18-981b-17b58fdb9dc9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:23.407161 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d13ee-5868-4b18-981b-17b58fdb9dc9 service placement] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:23.417569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1404/2827] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:18:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:18:26.131073 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e2248-60d6-46e0-b763-fcb6538f93e8 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:26.131544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e2248-60d6-46e0-b763-fcb6538f93e8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:26.131544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e2248-60d6-46e0-b763-fcb6538f93e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:26.150755 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-0d5e2248-60d6-46e0-b763-fcb6538f93e8 None None] MFA Rules not processed for user `8c21ea7d3cbb494886e60a4bd4f39145`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:18:26.152008 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0d5e2248-60d6-46e0-b763-fcb6538f93e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:26.159068 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1424/2828] 10.222.0.48 () {58 vars in 1009 bytes} [Fri Sep 9 21:18:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:18:26.171948 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d13ee-5868-4b18-981b-17b58fdb9dc9 service placement] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:26.183467 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d13ee-5868-4b18-981b-17b58fdb9dc9 service placement] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:26.188428 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f14ce-6233-4262-8355-83c407a3b040 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d28fa5d49d44120bd5eaec2d4e5fdbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:26.189025 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f14ce-6233-4262-8355-83c407a3b040 service swift] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:26.189302 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f14ce-6233-4262-8355-83c407a3b040 service swift] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:26.189568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f14ce-6233-4262-8355-83c407a3b040 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:26.197849 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295f14ce-6233-4262-8355-83c407a3b040 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:26.199501 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295f14ce-6233-4262-8355-83c407a3b040 service swift] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:26.208756 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1405/2829] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Sep 9 21:18:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 21:18:36.145810 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=29 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:36.204652 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=41 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:46.234511 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e2248-60d6-46e0-b763-fcb6538f93e8 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:46.236174 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0d5e2248-60d6-46e0-b763-fcb6538f93e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:46.309687 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e2248-60d6-46e0-b763-fcb6538f93e8 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:46.315230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3ece4-78f5-4fe1-a906-c0598553e480 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:46.315835 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3ece4-78f5-4fe1-a906-c0598553e480 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:46.316159 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3ece4-78f5-4fe1-a906-c0598553e480 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:46.316417 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3ece4-78f5-4fe1-a906-c0598553e480 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:46.318217 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cfa3ece4-78f5-4fe1-a906-c0598553e480 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:46.330434 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa3ece4-78f5-4fe1-a906-c0598553e480 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:46.332833 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa3ece4-78f5-4fe1-a906-c0598553e480 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:46.349125 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1425/2830] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:18:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:18:46.570338 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f14ce-6233-4262-8355-83c407a3b040 service swift] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:46.579849 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f14ce-6233-4262-8355-83c407a3b040 service swift] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:46.584429 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbd800-9aa6-48c7-8053-c5581d4508a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:46.585053 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbd800-9aa6-48c7-8053-c5581d4508a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:46.585330 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbd800-9aa6-48c7-8053-c5581d4508a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:46.585603 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbd800-9aa6-48c7-8053-c5581d4508a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:46.587390 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-1ecbd800-9aa6-48c7-8053-c5581d4508a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:46.668930 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecbd800-9aa6-48c7-8053-c5581d4508a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:46.670420 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecbd800-9aa6-48c7-8053-c5581d4508a2 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:46.689134 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1406/2831] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:18:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:18:56.343535 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=34 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:56.686398 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=41 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:02.383434 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3ece4-78f5-4fe1-a906-c0598553e480 service nova] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:19:02.392950 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3ece4-78f5-4fe1-a906-c0598553e480 service nova] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:19:02.396589 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b586a1f-2309-4a86-90a1-9a798eee0268 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:19:02.397179 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b586a1f-2309-4a86-90a1-9a798eee0268 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:02.397463 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b586a1f-2309-4a86-90a1-9a798eee0268 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:02.397736 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b586a1f-2309-4a86-90a1-9a798eee0268 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:02.399484 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7b586a1f-2309-4a86-90a1-9a798eee0268 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:02.469626 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b586a1f-2309-4a86-90a1-9a798eee0268 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:19:02.471174 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b586a1f-2309-4a86-90a1-9a798eee0268 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:19:02.481858 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1426/2832] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:19:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:19:03.112625 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbd800-9aa6-48c7-8053-c5581d4508a2 service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:19:03.121672 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbd800-9aa6-48c7-8053-c5581d4508a2 service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:19:03.125438 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7374f9e1-8cc5-4cba-bbe7-7d02e0c86cc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:19:03.126205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374f9e1-8cc5-4cba-bbe7-7d02e0c86cc7 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:03.126492 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374f9e1-8cc5-4cba-bbe7-7d02e0c86cc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:03.126764 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374f9e1-8cc5-4cba-bbe7-7d02e0c86cc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:03.136956 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7374f9e1-8cc5-4cba-bbe7-7d02e0c86cc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:19:03.138396 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7374f9e1-8cc5-4cba-bbe7-7d02e0c86cc7 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:19:03.148820 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1407/2833] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:19:03 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:19:06.702716 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b586a1f-2309-4a86-90a1-9a798eee0268 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:19:06.715210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b586a1f-2309-4a86-90a1-9a798eee0268 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:19:06.720287 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5431d1ce-186c-482a-aa4d-eb29d19b34bc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4f6de2065d4fbea3fa141da760f708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:19:06.720887 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431d1ce-186c-482a-aa4d-eb29d19b34bc service placement] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:06.721162 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431d1ce-186c-482a-aa4d-eb29d19b34bc service placement] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:06.721432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431d1ce-186c-482a-aa4d-eb29d19b34bc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:06.723954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5431d1ce-186c-482a-aa4d-eb29d19b34bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:06.736881 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5431d1ce-186c-482a-aa4d-eb29d19b34bc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:19:06.738621 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5431d1ce-186c-482a-aa4d-eb29d19b34bc service placement] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:19:06.749176 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1427/2834] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:19:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:19:11.659899 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7374f9e1-8cc5-4cba-bbe7-7d02e0c86cc7 service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:19:11.661492 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-7374f9e1-8cc5-4cba-bbe7-7d02e0c86cc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:11.677090 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7374f9e1-8cc5-4cba-bbe7-7d02e0c86cc7 service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:19:11.681553 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bbbe92-e773-4a35-ba26-f46bc543603c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:19:11.682166 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbbe92-e773-4a35-ba26-f46bc543603c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:11.682443 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbbe92-e773-4a35-ba26-f46bc543603c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:11.682719 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbbe92-e773-4a35-ba26-f46bc543603c admin admin] PATH_INFO: `/v3/users/f289a1bdd79b4ba0bf9499edb7729c9a` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:11.683894 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bbbe92-e773-4a35-ba26-f46bc543603c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f289a1bdd79b4ba0bf9499edb7729c9a)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:19:11.685821 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bbbe92-e773-4a35-ba26-f46bc543603c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:19:11.864761 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-33bbbe92-e773-4a35-ba26-f46bc543603c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f289a1bdd79b4ba0bf9499edb7729c9a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:19:11.865332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-33bbbe92-e773-4a35-ba26-f46bc543603c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f289a1bdd79b4ba0bf9499edb7729c9a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:19:11.872550 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-33bbbe92-e773-4a35-ba26-f46bc543603c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f289a1bdd79b4ba0bf9499edb7729c9a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:19:11.885566 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-33bbbe92-e773-4a35-ba26-f46bc543603c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f289a1bdd79b4ba0bf9499edb7729c9a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:19:11.886258 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-33bbbe92-e773-4a35-ba26-f46bc543603c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f289a1bdd79b4ba0bf9499edb7729c9a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:19:11.964774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1408/2835] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:19:11 2022] DELETE /identity/v3/users/f289a1bdd79b4ba0bf9499edb7729c9a => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:19:12.203900 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5431d1ce-186c-482a-aa4d-eb29d19b34bc service placement] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:19:12.215215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5431d1ce-186c-482a-aa4d-eb29d19b34bc service placement] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:19:12.273366 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efeb142-629e-4ad3-b8a3-180a0c34eb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:19:12.274004 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efeb142-629e-4ad3-b8a3-180a0c34eb66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:12.274285 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efeb142-629e-4ad3-b8a3-180a0c34eb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:12.274560 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efeb142-629e-4ad3-b8a3-180a0c34eb66 admin admin] PATH_INFO: `/v3/projects/c8403c10131d45b7a3f34ae872ac9e9e` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:12.275672 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efeb142-629e-4ad3-b8a3-180a0c34eb66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8403c10131d45b7a3f34ae872ac9e9e)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:19:12.277620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efeb142-629e-4ad3-b8a3-180a0c34eb66 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:19:12.405499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-0efeb142-629e-4ad3-b8a3-180a0c34eb66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8403c10131d45b7a3f34ae872ac9e9e', 'request_id': 'req-0efeb142-629e-4ad3-b8a3-180a0c34eb66'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:19:12.412472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-0efeb142-629e-4ad3-b8a3-180a0c34eb66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8403c10131d45b7a3f34ae872ac9e9e', 'request_id': 'req-0efeb142-629e-4ad3-b8a3-180a0c34eb66'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:19:12.513123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-0efeb142-629e-4ad3-b8a3-180a0c34eb66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8403c10131d45b7a3f34ae872ac9e9e', 'request_id': 'req-0efeb142-629e-4ad3-b8a3-180a0c34eb66'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:19:12.524877 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-0efeb142-629e-4ad3-b8a3-180a0c34eb66 admin admin] The token cache is being invalidate because project c8403c10131d45b7a3f34ae872ac9e9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:19:12.525240 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-0efeb142-629e-4ad3-b8a3-180a0c34eb66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:19:12.531352 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-0efeb142-629e-4ad3-b8a3-180a0c34eb66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:19:12.531352 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1428/2836] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:19:12 2022] DELETE /identity/v3/projects/c8403c10131d45b7a3f34ae872ac9e9e => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:19:12.531799 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bbbe92-e773-4a35-ba26-f46bc543603c admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:19:12.533217 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-33bbbe92-e773-4a35-ba26-f46bc543603c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:12.546877 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bbbe92-e773-4a35-ba26-f46bc543603c admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:19:12.603859 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d55886-1072-4d76-b288-ca52d7e2feb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:19:12.604520 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d55886-1072-4d76-b288-ca52d7e2feb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:12.604798 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d55886-1072-4d76-b288-ca52d7e2feb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:12.605069 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d55886-1072-4d76-b288-ca52d7e2feb7 admin admin] PATH_INFO: `/v3/users/3274028787bf4d9f9797906c9d1d9507` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:12.617954 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d55886-1072-4d76-b288-ca52d7e2feb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3274028787bf4d9f9797906c9d1d9507)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:19:12.619868 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d55886-1072-4d76-b288-ca52d7e2feb7 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:19:12.795930 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a6d55886-1072-4d76-b288-ca52d7e2feb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3274028787bf4d9f9797906c9d1d9507'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:19:12.796797 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a6d55886-1072-4d76-b288-ca52d7e2feb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3274028787bf4d9f9797906c9d1d9507'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:19:12.798874 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:12.804427 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a6d55886-1072-4d76-b288-ca52d7e2feb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3274028787bf4d9f9797906c9d1d9507'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:19:12.816919 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a6d55886-1072-4d76-b288-ca52d7e2feb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3274028787bf4d9f9797906c9d1d9507'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:19:12.817574 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a6d55886-1072-4d76-b288-ca52d7e2feb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3274028787bf4d9f9797906c9d1d9507'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:19:12.901540 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1409/2837] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:19:12 2022] DELETE /identity/v3/users/3274028787bf4d9f9797906c9d1d9507 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:19:13.127317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efeb142-629e-4ad3-b8a3-180a0c34eb66 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:19:13.128940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0efeb142-629e-4ad3-b8a3-180a0c34eb66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:13.142033 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efeb142-629e-4ad3-b8a3-180a0c34eb66 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:19:13.150580 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:13.198785 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50722d53-bdd7-4269-9c6b-db1374b35c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:19:13.199427 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50722d53-bdd7-4269-9c6b-db1374b35c78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:13.199707 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50722d53-bdd7-4269-9c6b-db1374b35c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:13.199978 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50722d53-bdd7-4269-9c6b-db1374b35c78 admin admin] PATH_INFO: `/v3/projects/07aa76b3c415436f83339df773c696fb` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:13.211517 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50722d53-bdd7-4269-9c6b-db1374b35c78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07aa76b3c415436f83339df773c696fb)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:19:13.213496 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50722d53-bdd7-4269-9c6b-db1374b35c78 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:19:13.295890 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-50722d53-bdd7-4269-9c6b-db1374b35c78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07aa76b3c415436f83339df773c696fb', 'request_id': 'req-50722d53-bdd7-4269-9c6b-db1374b35c78'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:19:13.302552 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-50722d53-bdd7-4269-9c6b-db1374b35c78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07aa76b3c415436f83339df773c696fb', 'request_id': 'req-50722d53-bdd7-4269-9c6b-db1374b35c78'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:19:13.359046 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-50722d53-bdd7-4269-9c6b-db1374b35c78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07aa76b3c415436f83339df773c696fb', 'request_id': 'req-50722d53-bdd7-4269-9c6b-db1374b35c78'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:19:13.369749 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-50722d53-bdd7-4269-9c6b-db1374b35c78 admin admin] The token cache is being invalidate because project 07aa76b3c415436f83339df773c696fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:19:13.370038 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-50722d53-bdd7-4269-9c6b-db1374b35c78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:19:13.370656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-50722d53-bdd7-4269-9c6b-db1374b35c78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:19:13.372244 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1429/2838] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:19:13 2022] DELETE /identity/v3/projects/07aa76b3c415436f83339df773c696fb => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:19:20.546289 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5a62d-c5c8-4050-a8a3-11311a33ceb2 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:20.546803 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5a62d-c5c8-4050-a8a3-11311a33ceb2 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:20.546919 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5a62d-c5c8-4050-a8a3-11311a33ceb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:20.567400 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-d1b5a62d-c5c8-4050-a8a3-11311a33ceb2 None None] MFA Rules not processed for user `56e50c46ac774417a50af986c5f87420`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:19:20.568634 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d1b5a62d-c5c8-4050-a8a3-11311a33ceb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:20.624505 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1410/2839] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:19:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:19:23.366690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=50,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:30.617707 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=36,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:36.117598 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50722d53-bdd7-4269-9c6b-db1374b35c78 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:19:36.119069 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-50722d53-bdd7-4269-9c6b-db1374b35c78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:36.187517 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50722d53-bdd7-4269-9c6b-db1374b35c78 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:19:36.192967 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746012b-3837-44c6-b9d7-44344714514a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4f6de2065d4fbea3fa141da760f708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:19:36.193565 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746012b-3837-44c6-b9d7-44344714514a service placement] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:36.193843 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746012b-3837-44c6-b9d7-44344714514a service placement] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:36.194116 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746012b-3837-44c6-b9d7-44344714514a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:36.195817 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e746012b-3837-44c6-b9d7-44344714514a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:36.207990 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e746012b-3837-44c6-b9d7-44344714514a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:19:36.209617 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e746012b-3837-44c6-b9d7-44344714514a service placement] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:19:36.219657 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1430/2840] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:19:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:19:46.215350 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=32 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:20:49.254126 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b5a62d-c5c8-4050-a8a3-11311a33ceb2 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:20:49.255522 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d1b5a62d-c5c8-4050-a8a3-11311a33ceb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:20:49.310527 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b5a62d-c5c8-4050-a8a3-11311a33ceb2 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:20:49.313989 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d174e3-08a0-445c-9413-d3b32ddbfaf7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:20:49.314573 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d174e3-08a0-445c-9413-d3b32ddbfaf7 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:20:49.314844 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d174e3-08a0-445c-9413-d3b32ddbfaf7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:20:49.315120 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d174e3-08a0-445c-9413-d3b32ddbfaf7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:20:49.316826 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d3d174e3-08a0-445c-9413-d3b32ddbfaf7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:20:49.385407 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d174e3-08a0-445c-9413-d3b32ddbfaf7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:20:49.386817 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d174e3-08a0-445c-9413-d3b32ddbfaf7 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:20:49.397767 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1411/2841] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:20:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:20:51.001560 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746012b-3837-44c6-b9d7-44344714514a service placement] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:20:51.003085 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e746012b-3837-44c6-b9d7-44344714514a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:20:51.060732 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746012b-3837-44c6-b9d7-44344714514a service placement] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:20:51.064272 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4e597-ceb9-4cd8-8d18-d775f6546ca5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:20:51.064870 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4e597-ceb9-4cd8-8d18-d775f6546ca5 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:20:51.065150 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4e597-ceb9-4cd8-8d18-d775f6546ca5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:20:51.065419 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4e597-ceb9-4cd8-8d18-d775f6546ca5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:20:51.067124 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-57c4e597-ceb9-4cd8-8d18-d775f6546ca5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:20:51.080192 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c4e597-ceb9-4cd8-8d18-d775f6546ca5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:20:51.081614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c4e597-ceb9-4cd8-8d18-d775f6546ca5 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:20:51.092726 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1431/2842] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:20:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:20:59.393020 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=54 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:21:01.087991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=26 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:21:06.979807 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9fa73-65de-45cd-a8b3-60a211689f01 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:21:06.980354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9fa73-65de-45cd-a8b3-60a211689f01 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:21:06.980461 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9fa73-65de-45cd-a8b3-60a211689f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:21:06.999698 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-50e9fa73-65de-45cd-a8b3-60a211689f01 None None] MFA Rules not processed for user `56e50c46ac774417a50af986c5f87420`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:21:07.000937 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-50e9fa73-65de-45cd-a8b3-60a211689f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:21:07.008289 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1412/2843] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:21:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:21:10.729827 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4e597-ceb9-4cd8-8d18-d775f6546ca5 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:21:10.732360 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-57c4e597-ceb9-4cd8-8d18-d775f6546ca5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:21:10.745499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4e597-ceb9-4cd8-8d18-d775f6546ca5 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:21:10.750718 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e47e9-6435-4d51-927c-2224c4a38bda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:21:10.751508 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e47e9-6435-4d51-927c-2224c4a38bda service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:21:10.751785 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e47e9-6435-4d51-927c-2224c4a38bda service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:21:10.752271 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e47e9-6435-4d51-927c-2224c4a38bda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:21:10.754278 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2d2e47e9-6435-4d51-927c-2224c4a38bda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:21:10.772905 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e47e9-6435-4d51-927c-2224c4a38bda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:21:10.775151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e47e9-6435-4d51-927c-2224c4a38bda service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:21:10.788022 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1432/2844] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:21:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:21:16.994141 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:21:20.781104 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=6 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:21:21.439629 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbd57c-1ea3-4967-beaf-810e7208acda None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:21:21.440122 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbd57c-1ea3-4967-beaf-810e7208acda None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:21:21.440261 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbd57c-1ea3-4967-beaf-810e7208acda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:21:21.459887 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-61cbd57c-1ea3-4967-beaf-810e7208acda None None] MFA Rules not processed for user `8c21ea7d3cbb494886e60a4bd4f39145`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:21:21.461129 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-61cbd57c-1ea3-4967-beaf-810e7208acda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:21:21.515759 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1413/2845] 10.222.0.48 () {58 vars in 967 bytes} [Fri Sep 9 21:21:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:21:21.555365 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e47e9-6435-4d51-927c-2224c4a38bda service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:21:21.557383 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2d2e47e9-6435-4d51-927c-2224c4a38bda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:21:21.625239 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e47e9-6435-4d51-927c-2224c4a38bda service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:21:21.630091 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd7ec3-01c6-4374-a1fd-533ceba43640 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d28fa5d49d44120bd5eaec2d4e5fdbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:21:21.630686 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd7ec3-01c6-4374-a1fd-533ceba43640 service swift] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:21:21.630968 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd7ec3-01c6-4374-a1fd-533ceba43640 service swift] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:21:21.631242 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd7ec3-01c6-4374-a1fd-533ceba43640 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:21:21.639081 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd7ec3-01c6-4374-a1fd-533ceba43640 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:21:21.640703 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd7ec3-01c6-4374-a1fd-533ceba43640 service swift] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:21:21.649607 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1433/2846] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Sep 9 21:21:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 21:21:26.494454 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cbd57c-1ea3-4967-beaf-810e7208acda None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:21:26.504538 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cbd57c-1ea3-4967-beaf-810e7208acda None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:21:26.509585 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43d5ae-f1ad-4abd-8913-50180bf8186e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4f6de2065d4fbea3fa141da760f708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:21:26.510205 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43d5ae-f1ad-4abd-8913-50180bf8186e service placement] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:21:26.510479 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43d5ae-f1ad-4abd-8913-50180bf8186e service placement] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:21:26.510750 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43d5ae-f1ad-4abd-8913-50180bf8186e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:21:26.512524 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-9b43d5ae-f1ad-4abd-8913-50180bf8186e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:21:26.524671 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43d5ae-f1ad-4abd-8913-50180bf8186e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:21:26.526881 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43d5ae-f1ad-4abd-8913-50180bf8186e service placement] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:21:26.538195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1414/2847] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:21:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:21:30.271984 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd7ec3-01c6-4374-a1fd-533ceba43640 service swift] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:21:30.273468 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3ddd7ec3-01c6-4374-a1fd-533ceba43640 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:21:30.342544 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd7ec3-01c6-4374-a1fd-533ceba43640 service swift] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:21:30.347103 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6308dd98-a5c6-4ec3-b6fa-877e4a0ebfcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:21:30.347724 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308dd98-a5c6-4ec3-b6fa-877e4a0ebfcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:21:30.348003 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308dd98-a5c6-4ec3-b6fa-877e4a0ebfcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:21:30.348312 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308dd98-a5c6-4ec3-b6fa-877e4a0ebfcd admin admin] PATH_INFO: `/v3/users/fa04bcfffac2473baad2953f1c64eb95` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:21:30.349423 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6308dd98-a5c6-4ec3-b6fa-877e4a0ebfcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa04bcfffac2473baad2953f1c64eb95)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:21:30.351343 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6308dd98-a5c6-4ec3-b6fa-877e4a0ebfcd admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:21:30.524259 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6308dd98-a5c6-4ec3-b6fa-877e4a0ebfcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa04bcfffac2473baad2953f1c64eb95'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:21:30.524861 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6308dd98-a5c6-4ec3-b6fa-877e4a0ebfcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa04bcfffac2473baad2953f1c64eb95'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:21:30.526072 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6308dd98-a5c6-4ec3-b6fa-877e4a0ebfcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa04bcfffac2473baad2953f1c64eb95'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:21:30.532887 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6308dd98-a5c6-4ec3-b6fa-877e4a0ebfcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa04bcfffac2473baad2953f1c64eb95'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:21:30.544687 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6308dd98-a5c6-4ec3-b6fa-877e4a0ebfcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa04bcfffac2473baad2953f1c64eb95'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:21:30.607893 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1434/2848] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:21:30 2022] DELETE /identity/v3/users/fa04bcfffac2473baad2953f1c64eb95 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:21:30.839520 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43d5ae-f1ad-4abd-8913-50180bf8186e service placement] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:21:30.849975 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43d5ae-f1ad-4abd-8913-50180bf8186e service placement] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:21:30.909715 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471f632-51a5-4c0f-bcbb-ebb480434379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:21:30.910644 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471f632-51a5-4c0f-bcbb-ebb480434379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:21:30.911911 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471f632-51a5-4c0f-bcbb-ebb480434379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:21:30.912241 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471f632-51a5-4c0f-bcbb-ebb480434379 admin admin] PATH_INFO: `/v3/projects/a1261e5e82414293b96ce77482498a89` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:21:30.913419 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d471f632-51a5-4c0f-bcbb-ebb480434379 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1261e5e82414293b96ce77482498a89)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:21:30.915372 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d471f632-51a5-4c0f-bcbb-ebb480434379 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:21:31.034384 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-d471f632-51a5-4c0f-bcbb-ebb480434379 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1261e5e82414293b96ce77482498a89', 'request_id': 'req-d471f632-51a5-4c0f-bcbb-ebb480434379'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:21:31.094517 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-d471f632-51a5-4c0f-bcbb-ebb480434379 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1261e5e82414293b96ce77482498a89', 'request_id': 'req-d471f632-51a5-4c0f-bcbb-ebb480434379'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:21:31.106031 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-d471f632-51a5-4c0f-bcbb-ebb480434379 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1261e5e82414293b96ce77482498a89', 'request_id': 'req-d471f632-51a5-4c0f-bcbb-ebb480434379'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:21:31.113086 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-d471f632-51a5-4c0f-bcbb-ebb480434379 admin admin] The token cache is being invalidate because project a1261e5e82414293b96ce77482498a89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:21:31.113377 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-d471f632-51a5-4c0f-bcbb-ebb480434379 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:21:31.114005 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-d471f632-51a5-4c0f-bcbb-ebb480434379 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:21:31.115458 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1415/2849] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:21:30 2022] DELETE /identity/v3/projects/a1261e5e82414293b96ce77482498a89 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:21:31.119465 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6308dd98-a5c6-4ec3-b6fa-877e4a0ebfcd admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:21:31.120954 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6308dd98-a5c6-4ec3-b6fa-877e4a0ebfcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:21:31.133734 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6308dd98-a5c6-4ec3-b6fa-877e4a0ebfcd admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:21:31.138572 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:21:31.191079 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7339336-78e8-4c7b-b7fa-9373004730a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:21:31.191838 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7339336-78e8-4c7b-b7fa-9373004730a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:21:31.192571 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7339336-78e8-4c7b-b7fa-9373004730a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:21:31.192811 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7339336-78e8-4c7b-b7fa-9373004730a2 admin admin] PATH_INFO: `/v3/users/ffd2599d3dcf4b708766cf963f73616d` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:21:31.193933 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7339336-78e8-4c7b-b7fa-9373004730a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffd2599d3dcf4b708766cf963f73616d)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:21:31.195836 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7339336-78e8-4c7b-b7fa-9373004730a2 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:21:31.346452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e7339336-78e8-4c7b-b7fa-9373004730a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffd2599d3dcf4b708766cf963f73616d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:21:31.347095 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e7339336-78e8-4c7b-b7fa-9373004730a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffd2599d3dcf4b708766cf963f73616d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:21:31.347610 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e7339336-78e8-4c7b-b7fa-9373004730a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffd2599d3dcf4b708766cf963f73616d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:21:31.355112 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e7339336-78e8-4c7b-b7fa-9373004730a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffd2599d3dcf4b708766cf963f73616d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:21:31.366891 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e7339336-78e8-4c7b-b7fa-9373004730a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffd2599d3dcf4b708766cf963f73616d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:21:31.436014 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1435/2850] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:21:31 2022] DELETE /identity/v3/users/ffd2599d3dcf4b708766cf963f73616d => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:21:31.666967 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471f632-51a5-4c0f-bcbb-ebb480434379 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:21:31.668488 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d471f632-51a5-4c0f-bcbb-ebb480434379 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:21:31.682733 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471f632-51a5-4c0f-bcbb-ebb480434379 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:21:31.702637 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:21:31.741004 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b09d9-fd42-4066-91b9-d89c6e83dc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:21:31.741686 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b09d9-fd42-4066-91b9-d89c6e83dc39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:21:31.741965 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b09d9-fd42-4066-91b9-d89c6e83dc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:21:31.742239 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b09d9-fd42-4066-91b9-d89c6e83dc39 admin admin] PATH_INFO: `/v3/projects/ca8416bca7fb45a3a30c90a17dea4c86` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:21:31.743361 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02b09d9-fd42-4066-91b9-d89c6e83dc39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca8416bca7fb45a3a30c90a17dea4c86)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:21:31.745834 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02b09d9-fd42-4066-91b9-d89c6e83dc39 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:21:31.875318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f02b09d9-fd42-4066-91b9-d89c6e83dc39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca8416bca7fb45a3a30c90a17dea4c86', 'request_id': 'req-f02b09d9-fd42-4066-91b9-d89c6e83dc39'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:21:31.954332 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f02b09d9-fd42-4066-91b9-d89c6e83dc39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca8416bca7fb45a3a30c90a17dea4c86', 'request_id': 'req-f02b09d9-fd42-4066-91b9-d89c6e83dc39'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:21:31.961187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f02b09d9-fd42-4066-91b9-d89c6e83dc39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca8416bca7fb45a3a30c90a17dea4c86', 'request_id': 'req-f02b09d9-fd42-4066-91b9-d89c6e83dc39'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:21:31.971630 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f02b09d9-fd42-4066-91b9-d89c6e83dc39 admin admin] The token cache is being invalidate because project ca8416bca7fb45a3a30c90a17dea4c86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:21:31.974476 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f02b09d9-fd42-4066-91b9-d89c6e83dc39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:21:31.974476 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f02b09d9-fd42-4066-91b9-d89c6e83dc39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:21:31.974476 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1416/2851] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:21:31 2022] DELETE /identity/v3/projects/ca8416bca7fb45a3a30c90a17dea4c86 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:21:41.432656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:21:41.968646 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=41,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:22:30.698218 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7339336-78e8-4c7b-b7fa-9373004730a2 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:22:30.700430 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e7339336-78e8-4c7b-b7fa-9373004730a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:22:30.777445 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7339336-78e8-4c7b-b7fa-9373004730a2 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:22:30.841171 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e81d7d7-f6b7-4aaf-ba52-48edf0675e20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:22:30.841768 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81d7d7-f6b7-4aaf-ba52-48edf0675e20 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:22:30.842049 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81d7d7-f6b7-4aaf-ba52-48edf0675e20 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:22:30.842325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81d7d7-f6b7-4aaf-ba52-48edf0675e20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:22:30.844061 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3e81d7d7-f6b7-4aaf-ba52-48edf0675e20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:22:30.901775 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e81d7d7-f6b7-4aaf-ba52-48edf0675e20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:22:30.903375 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e81d7d7-f6b7-4aaf-ba52-48edf0675e20 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:22:30.912472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1436/2852] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:22:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:22:40.908319 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=84 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:22:46.887293 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b09d9-fd42-4066-91b9-d89c6e83dc39 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:22:46.889258 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f02b09d9-fd42-4066-91b9-d89c6e83dc39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:22:46.916881 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b09d9-fd42-4066-91b9-d89c6e83dc39 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:22:46.916881 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40041322-96bb-4433-99cb-1db0ca7afbc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:22:46.916881 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40041322-96bb-4433-99cb-1db0ca7afbc5 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:22:46.916881 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40041322-96bb-4433-99cb-1db0ca7afbc5 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:22:46.917297 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40041322-96bb-4433-99cb-1db0ca7afbc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:22:46.917297 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-40041322-96bb-4433-99cb-1db0ca7afbc5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:22:46.928828 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40041322-96bb-4433-99cb-1db0ca7afbc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:22:46.930433 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40041322-96bb-4433-99cb-1db0ca7afbc5 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:22:46.941865 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1417/2853] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:22:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:22:51.635025 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381d454-b135-451c-8872-4b3f10d3b551 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:22:51.635537 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381d454-b135-451c-8872-4b3f10d3b551 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:22:51.635656 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381d454-b135-451c-8872-4b3f10d3b551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:22:51.655666 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c381d454-b135-451c-8872-4b3f10d3b551 None None] MFA Rules not processed for user `56e50c46ac774417a50af986c5f87420`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:22:51.656894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c381d454-b135-451c-8872-4b3f10d3b551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:22:51.663704 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1437/2854] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:22:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:22:56.937754 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=6 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:23:01.649492 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:23:07.671568 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40041322-96bb-4433-99cb-1db0ca7afbc5 service nova] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:23:07.682985 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40041322-96bb-4433-99cb-1db0ca7afbc5 service nova] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:23:07.688705 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357978c1-5db3-433a-aead-683f8ec2fbb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:23:07.689315 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357978c1-5db3-433a-aead-683f8ec2fbb0 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:23:07.689603 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357978c1-5db3-433a-aead-683f8ec2fbb0 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:23:07.689877 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357978c1-5db3-433a-aead-683f8ec2fbb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:23:07.691633 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-357978c1-5db3-433a-aead-683f8ec2fbb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:23:07.704915 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357978c1-5db3-433a-aead-683f8ec2fbb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:23:07.706488 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357978c1-5db3-433a-aead-683f8ec2fbb0 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:23:07.717785 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1418/2855] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:23:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:23:17.712758 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=6 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:24:37.177904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381d454-b135-451c-8872-4b3f10d3b551 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:24:37.179350 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c381d454-b135-451c-8872-4b3f10d3b551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:24:37.350293 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381d454-b135-451c-8872-4b3f10d3b551 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:24:37.355358 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07a6665-d768-4706-8b6e-e63c2fcbb3ea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4f6de2065d4fbea3fa141da760f708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:24:37.355958 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a6665-d768-4706-8b6e-e63c2fcbb3ea service placement] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:24:37.356901 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a6665-d768-4706-8b6e-e63c2fcbb3ea service placement] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:24:37.357188 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a6665-d768-4706-8b6e-e63c2fcbb3ea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:24:37.358931 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a07a6665-d768-4706-8b6e-e63c2fcbb3ea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:24:37.370547 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07a6665-d768-4706-8b6e-e63c2fcbb3ea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:24:37.372306 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07a6665-d768-4706-8b6e-e63c2fcbb3ea service placement] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:24:37.382455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1438/2856] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:24:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:24:44.377784 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357978c1-5db3-433a-aead-683f8ec2fbb0 service nova] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:24:44.379171 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-357978c1-5db3-433a-aead-683f8ec2fbb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:24:44.484619 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357978c1-5db3-433a-aead-683f8ec2fbb0 service nova] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:24:44.488303 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea1426-beba-4e85-9dee-0e486dab0ef0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:24:44.488901 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea1426-beba-4e85-9dee-0e486dab0ef0 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:24:44.489178 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea1426-beba-4e85-9dee-0e486dab0ef0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:24:44.489449 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea1426-beba-4e85-9dee-0e486dab0ef0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:24:44.491175 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-33ea1426-beba-4e85-9dee-0e486dab0ef0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:24:44.572791 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea1426-beba-4e85-9dee-0e486dab0ef0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:24:44.574131 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea1426-beba-4e85-9dee-0e486dab0ef0 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:24:44.594955 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1419/2857] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:24:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:24:47.378127 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=63 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:24:48.222359 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07a6665-d768-4706-8b6e-e63c2fcbb3ea service placement] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:24:48.224130 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a07a6665-d768-4706-8b6e-e63c2fcbb3ea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:24:48.238396 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07a6665-d768-4706-8b6e-e63c2fcbb3ea service placement] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:24:48.242987 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af111ffa-33ce-467b-a900-4471b61521d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:24:48.243701 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af111ffa-33ce-467b-a900-4471b61521d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:24:48.244012 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af111ffa-33ce-467b-a900-4471b61521d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:24:48.247134 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af111ffa-33ce-467b-a900-4471b61521d4 admin admin] PATH_INFO: `/v3/users/56e50c46ac774417a50af986c5f87420` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:24:48.247134 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af111ffa-33ce-467b-a900-4471b61521d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56e50c46ac774417a50af986c5f87420)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:24:48.247426 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af111ffa-33ce-467b-a900-4471b61521d4 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:24:48.457263 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-af111ffa-33ce-467b-a900-4471b61521d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56e50c46ac774417a50af986c5f87420'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:24:48.457918 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-af111ffa-33ce-467b-a900-4471b61521d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56e50c46ac774417a50af986c5f87420'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:24:48.458521 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-af111ffa-33ce-467b-a900-4471b61521d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56e50c46ac774417a50af986c5f87420'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:24:48.466276 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-af111ffa-33ce-467b-a900-4471b61521d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56e50c46ac774417a50af986c5f87420'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:24:48.478053 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-af111ffa-33ce-467b-a900-4471b61521d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56e50c46ac774417a50af986c5f87420'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:24:48.551550 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1439/2858] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:24:48 2022] DELETE /identity/v3/users/56e50c46ac774417a50af986c5f87420 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:24:48.787811 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea1426-beba-4e85-9dee-0e486dab0ef0 service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:24:48.804200 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea1426-beba-4e85-9dee-0e486dab0ef0 service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:24:48.830136 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:24:48.856471 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9a36c-0f18-4675-a3d7-2cf7e7b0f2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:24:48.857158 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9a36c-0f18-4675-a3d7-2cf7e7b0f2dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:24:48.857439 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9a36c-0f18-4675-a3d7-2cf7e7b0f2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:24:48.857708 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9a36c-0f18-4675-a3d7-2cf7e7b0f2dd admin admin] PATH_INFO: `/v3/projects/1a518e979c944b3ab2a1245b3cfd246f` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:24:48.858852 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9a36c-0f18-4675-a3d7-2cf7e7b0f2dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a518e979c944b3ab2a1245b3cfd246f)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:24:48.860834 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9a36c-0f18-4675-a3d7-2cf7e7b0f2dd admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:24:48.962774 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e8c9a36c-0f18-4675-a3d7-2cf7e7b0f2dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a518e979c944b3ab2a1245b3cfd246f', 'request_id': 'req-e8c9a36c-0f18-4675-a3d7-2cf7e7b0f2dd'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:24:49.023823 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e8c9a36c-0f18-4675-a3d7-2cf7e7b0f2dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a518e979c944b3ab2a1245b3cfd246f', 'request_id': 'req-e8c9a36c-0f18-4675-a3d7-2cf7e7b0f2dd'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:24:49.031521 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e8c9a36c-0f18-4675-a3d7-2cf7e7b0f2dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a518e979c944b3ab2a1245b3cfd246f', 'request_id': 'req-e8c9a36c-0f18-4675-a3d7-2cf7e7b0f2dd'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:24:49.044831 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e8c9a36c-0f18-4675-a3d7-2cf7e7b0f2dd admin admin] The token cache is being invalidate because project 1a518e979c944b3ab2a1245b3cfd246f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:24:49.045188 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e8c9a36c-0f18-4675-a3d7-2cf7e7b0f2dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:24:49.045844 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e8c9a36c-0f18-4675-a3d7-2cf7e7b0f2dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:24:49.047323 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1420/2859] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:24:48 2022] DELETE /identity/v3/projects/1a518e979c944b3ab2a1245b3cfd246f => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:24:49.052316 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ae7aa-898e-40a6-be14-7d013be40833 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:24:49.052674 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ae7aa-898e-40a6-be14-7d013be40833 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:24:49.052959 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ae7aa-898e-40a6-be14-7d013be40833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:24:49.111439 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d02ae7aa-898e-40a6-be14-7d013be40833 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:24:49.112673 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d02ae7aa-898e-40a6-be14-7d013be40833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:24:49.170338 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1440/2860] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:24:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:24:49.178003 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4fd24-e63e-4178-b3b2-3e1d39c1d239 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:24:49.178288 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4fd24-e63e-4178-b3b2-3e1d39c1d239 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:24:49.178562 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4fd24-e63e-4178-b3b2-3e1d39c1d239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:24:49.198293 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-cdf4fd24-e63e-4178-b3b2-3e1d39c1d239 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:24:49.199482 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-cdf4fd24-e63e-4178-b3b2-3e1d39c1d239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:24:49.208753 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1421/2861] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:24:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:24:49.213124 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ae7aa-898e-40a6-be14-7d013be40833 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:24:49.223210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ae7aa-898e-40a6-be14-7d013be40833 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:24:49.227759 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11d3a8-bf70-44fc-a0df-6dd04d136ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:24:49.228377 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11d3a8-bf70-44fc-a0df-6dd04d136ab6 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:24:49.228652 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11d3a8-bf70-44fc-a0df-6dd04d136ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:24:49.228919 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11d3a8-bf70-44fc-a0df-6dd04d136ab6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:24:49.229563 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11d3a8-bf70-44fc-a0df-6dd04d136ab6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:24:49.229861 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11d3a8-bf70-44fc-a0df-6dd04d136ab6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:24:49.231981 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11d3a8-bf70-44fc-a0df-6dd04d136ab6 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:24:49.243146 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1441/2862] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:24:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:24:49.247839 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4fd24-e63e-4178-b3b2-3e1d39c1d239 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:24:49.256801 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4fd24-e63e-4178-b3b2-3e1d39c1d239 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:24:49.261370 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e312d-b589-48de-ab96-bf8b4fe2455f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:24:49.261952 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e312d-b589-48de-ab96-bf8b4fe2455f admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:24:49.262276 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e312d-b589-48de-ab96-bf8b4fe2455f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:24:49.262547 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e312d-b589-48de-ab96-bf8b4fe2455f admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:24:49.263183 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e312d-b589-48de-ab96-bf8b4fe2455f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:24:49.265104 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e312d-b589-48de-ab96-bf8b4fe2455f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:24:49.345331 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1422/2863] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:24:49 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:24:49.353966 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11d3a8-bf70-44fc-a0df-6dd04d136ab6 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:24:49.365175 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11d3a8-bf70-44fc-a0df-6dd04d136ab6 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:24:49.392192 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:24:49.422597 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6ac21-28cb-4716-b2bf-6f7fd320b5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:24:49.423195 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6ac21-28cb-4716-b2bf-6f7fd320b5a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:24:49.423474 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6ac21-28cb-4716-b2bf-6f7fd320b5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:24:49.423744 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6ac21-28cb-4716-b2bf-6f7fd320b5a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:24:49.424447 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6ac21-28cb-4716-b2bf-6f7fd320b5a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:24:49.426387 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6ac21-28cb-4716-b2bf-6f7fd320b5a1 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:24:49.526921 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1442/2864] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:24:49 2022] POST /identity/v3/users => generated 292 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:24:49.531695 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e312d-b589-48de-ab96-bf8b4fe2455f admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:24:49.541997 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e312d-b589-48de-ab96-bf8b4fe2455f admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:24:49.547782 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f65b2c-4a21-443f-8b93-84a6fb1b7f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:24:49.548404 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f65b2c-4a21-443f-8b93-84a6fb1b7f82 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:24:49.548690 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f65b2c-4a21-443f-8b93-84a6fb1b7f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:24:49.548961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f65b2c-4a21-443f-8b93-84a6fb1b7f82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:24:49.549600 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f65b2c-4a21-443f-8b93-84a6fb1b7f82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:24:49.551195 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f65b2c-4a21-443f-8b93-84a6fb1b7f82 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:24:49.561020 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1423/2865] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:24:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:24:49.565090 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6ac21-28cb-4716-b2bf-6f7fd320b5a1 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:24:49.575226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6ac21-28cb-4716-b2bf-6f7fd320b5a1 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:24:49.579849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7603685c-0d1f-441b-b432-63a852c1e688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:24:49.580472 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603685c-0d1f-441b-b432-63a852c1e688 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:24:49.580761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603685c-0d1f-441b-b432-63a852c1e688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:24:49.581037 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603685c-0d1f-441b-b432-63a852c1e688 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:24:49.581680 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7603685c-0d1f-441b-b432-63a852c1e688 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:24:49.583264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7603685c-0d1f-441b-b432-63a852c1e688 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:24:49.594530 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1443/2866] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:24:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:24:49.598549 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f65b2c-4a21-443f-8b93-84a6fb1b7f82 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:24:49.609184 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f65b2c-4a21-443f-8b93-84a6fb1b7f82 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:24:49.614048 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750867c-39df-4b0d-b67f-1ce6c21c3d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:24:49.614680 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750867c-39df-4b0d-b67f-1ce6c21c3d1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:24:49.614958 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750867c-39df-4b0d-b67f-1ce6c21c3d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:24:49.615230 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750867c-39df-4b0d-b67f-1ce6c21c3d1d admin admin] PATH_INFO: `/v3/projects/40066b66bdce448ea767fab6c22ce2bc/users/0646557ce9b64228a449ce5f208be980/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:24:49.629421 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d750867c-39df-4b0d-b67f-1ce6c21c3d1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=40066b66bdce448ea767fab6c22ce2bc, user_id=0646557ce9b64228a449ce5f208be980, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:24:49.632033 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d750867c-39df-4b0d-b67f-1ce6c21c3d1d admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:24:49.729773 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1424/2867] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:24:49 2022] PUT /identity/v3/projects/40066b66bdce448ea767fab6c22ce2bc/users/0646557ce9b64228a449ce5f208be980/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:24:49.740799 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d04347-48bd-4c11-9dad-79d18b829995 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:24:49.741087 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d04347-48bd-4c11-9dad-79d18b829995 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:24:49.741362 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d04347-48bd-4c11-9dad-79d18b829995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:24:49.769307 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e2d04347-48bd-4c11-9dad-79d18b829995 None None] MFA Rules not processed for user `0646557ce9b64228a449ce5f208be980`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:24:49.770462 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e2d04347-48bd-4c11-9dad-79d18b829995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:24:49.834762 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1444/2868] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:24:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:24:49.847070 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750867c-39df-4b0d-b67f-1ce6c21c3d1d admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:24:49.848808 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d750867c-39df-4b0d-b67f-1ce6c21c3d1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:24:49.907106 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:24:49.928892 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750867c-39df-4b0d-b67f-1ce6c21c3d1d admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:24:49.934435 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149442c7-287f-4102-bef7-1e753e1f8996 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:24:49.935057 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149442c7-287f-4102-bef7-1e753e1f8996 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:24:49.935334 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149442c7-287f-4102-bef7-1e753e1f8996 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:24:49.935605 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149442c7-287f-4102-bef7-1e753e1f8996 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:24:49.944378 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149442c7-287f-4102-bef7-1e753e1f8996 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:24:49.946075 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149442c7-287f-4102-bef7-1e753e1f8996 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:24:49.966866 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1425/2869] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:24:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:24:59.831061 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:00.001219 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=10 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:00.156688 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d04347-48bd-4c11-9dad-79d18b829995 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:00.216487 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d04347-48bd-4c11-9dad-79d18b829995 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:00.220952 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e181686-eb1d-4a3f-a080-998901a28bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:00.221567 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e181686-eb1d-4a3f-a080-998901a28bcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:00.221847 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e181686-eb1d-4a3f-a080-998901a28bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:00.222118 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e181686-eb1d-4a3f-a080-998901a28bcd admin admin] PATH_INFO: `/v3/users/0646557ce9b64228a449ce5f208be980` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:00.223250 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e181686-eb1d-4a3f-a080-998901a28bcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0646557ce9b64228a449ce5f208be980)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:00.225204 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e181686-eb1d-4a3f-a080-998901a28bcd admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:00.421781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8e181686-eb1d-4a3f-a080-998901a28bcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0646557ce9b64228a449ce5f208be980'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:00.422453 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8e181686-eb1d-4a3f-a080-998901a28bcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0646557ce9b64228a449ce5f208be980'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:00.423076 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8e181686-eb1d-4a3f-a080-998901a28bcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0646557ce9b64228a449ce5f208be980'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:00.430007 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8e181686-eb1d-4a3f-a080-998901a28bcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0646557ce9b64228a449ce5f208be980'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:00.441950 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8e181686-eb1d-4a3f-a080-998901a28bcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0646557ce9b64228a449ce5f208be980'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:00.522775 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1445/2870] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:25:00 2022] DELETE /identity/v3/users/0646557ce9b64228a449ce5f208be980 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:00.527234 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d28e39-dd66-4207-8184-2703d2b99535 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:00.527516 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d28e39-dd66-4207-8184-2703d2b99535 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:00.527786 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d28e39-dd66-4207-8184-2703d2b99535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:00.546958 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-97d28e39-dd66-4207-8184-2703d2b99535 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:00.548174 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-97d28e39-dd66-4207-8184-2703d2b99535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:00.606455 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1426/2871] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:00.616185 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e181686-eb1d-4a3f-a080-998901a28bcd admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:00.617587 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8e181686-eb1d-4a3f-a080-998901a28bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:00.673738 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e181686-eb1d-4a3f-a080-998901a28bcd admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:00.677293 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d893016-d3cf-4912-add4-01516b96fc6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:00.677892 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d893016-d3cf-4912-add4-01516b96fc6f service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:00.678174 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d893016-d3cf-4912-add4-01516b96fc6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:00.678450 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d893016-d3cf-4912-add4-01516b96fc6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:00.687847 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d893016-d3cf-4912-add4-01516b96fc6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:00.689323 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d893016-d3cf-4912-add4-01516b96fc6f service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:00.698586 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1446/2872] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:01.096043 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d28e39-dd66-4207-8184-2703d2b99535 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:01.097547 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-97d28e39-dd66-4207-8184-2703d2b99535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:01.113175 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d28e39-dd66-4207-8184-2703d2b99535 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:01.119074 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2ab2a-cbe3-47e7-8f41-ff512c5908ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:01.119845 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2ab2a-cbe3-47e7-8f41-ff512c5908ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:01.120023 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2ab2a-cbe3-47e7-8f41-ff512c5908ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:01.120425 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2ab2a-cbe3-47e7-8f41-ff512c5908ed admin admin] PATH_INFO: `/v3/projects/40066b66bdce448ea767fab6c22ce2bc` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:01.121554 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e2ab2a-cbe3-47e7-8f41-ff512c5908ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=40066b66bdce448ea767fab6c22ce2bc)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:01.124040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e2ab2a-cbe3-47e7-8f41-ff512c5908ed admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:01.240660 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-91e2ab2a-cbe3-47e7-8f41-ff512c5908ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40066b66bdce448ea767fab6c22ce2bc', 'request_id': 'req-91e2ab2a-cbe3-47e7-8f41-ff512c5908ed'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:01.303677 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-91e2ab2a-cbe3-47e7-8f41-ff512c5908ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40066b66bdce448ea767fab6c22ce2bc', 'request_id': 'req-91e2ab2a-cbe3-47e7-8f41-ff512c5908ed'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:01.310871 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-91e2ab2a-cbe3-47e7-8f41-ff512c5908ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40066b66bdce448ea767fab6c22ce2bc', 'request_id': 'req-91e2ab2a-cbe3-47e7-8f41-ff512c5908ed'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:01.321174 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-91e2ab2a-cbe3-47e7-8f41-ff512c5908ed admin admin] The token cache is being invalidate because project 40066b66bdce448ea767fab6c22ce2bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:25:01.321504 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-91e2ab2a-cbe3-47e7-8f41-ff512c5908ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:01.322097 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-91e2ab2a-cbe3-47e7-8f41-ff512c5908ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:01.323591 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1427/2873] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:25:01 2022] DELETE /identity/v3/projects/40066b66bdce448ea767fab6c22ce2bc => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:01.328484 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53244d5-394a-49e8-8b64-c4a63a9f5840 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:01.329084 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53244d5-394a-49e8-8b64-c4a63a9f5840 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:01.329084 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53244d5-394a-49e8-8b64-c4a63a9f5840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:01.350317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d53244d5-394a-49e8-8b64-c4a63a9f5840 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:01.351516 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d53244d5-394a-49e8-8b64-c4a63a9f5840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:01.369110 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:01.410214 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1447/2874] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:01.420151 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ea899-81f2-4474-844c-098ccf864f75 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:01.420429 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ea899-81f2-4474-844c-098ccf864f75 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:01.420696 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ea899-81f2-4474-844c-098ccf864f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:01.440284 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-381ea899-81f2-4474-844c-098ccf864f75 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:01.441481 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-381ea899-81f2-4474-844c-098ccf864f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:01.449295 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1428/2875] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:01.453616 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53244d5-394a-49e8-8b64-c4a63a9f5840 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:01.463551 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53244d5-394a-49e8-8b64-c4a63a9f5840 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:01.468584 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78687a1-629e-40d0-bd60-2cbb14221374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:01.469175 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78687a1-629e-40d0-bd60-2cbb14221374 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:01.469449 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78687a1-629e-40d0-bd60-2cbb14221374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:01.469716 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78687a1-629e-40d0-bd60-2cbb14221374 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:01.470355 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78687a1-629e-40d0-bd60-2cbb14221374 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:25:01.470651 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78687a1-629e-40d0-bd60-2cbb14221374 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:01.472258 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78687a1-629e-40d0-bd60-2cbb14221374 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:01.484549 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1448/2876] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:25:01.488364 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ea899-81f2-4474-844c-098ccf864f75 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:01.501960 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ea899-81f2-4474-844c-098ccf864f75 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:01.502129 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc5c99-704f-4bc4-ab4f-08cc0abc79f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:01.502640 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc5c99-704f-4bc4-ab4f-08cc0abc79f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:01.502917 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc5c99-704f-4bc4-ab4f-08cc0abc79f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:01.503184 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc5c99-704f-4bc4-ab4f-08cc0abc79f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:01.503809 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc5c99-704f-4bc4-ab4f-08cc0abc79f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:01.505723 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc5c99-704f-4bc4-ab4f-08cc0abc79f8 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:01.585143 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1429/2877] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:25:01 2022] POST /identity/v3/projects => generated 405 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:01.589536 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78687a1-629e-40d0-bd60-2cbb14221374 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:01.599875 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78687a1-629e-40d0-bd60-2cbb14221374 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:01.654529 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99e5ba-0c19-4787-9ed8-69b454750f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:01.655128 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99e5ba-0c19-4787-9ed8-69b454750f9b admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:01.655413 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99e5ba-0c19-4787-9ed8-69b454750f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:01.655684 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99e5ba-0c19-4787-9ed8-69b454750f9b admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:01.656352 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f99e5ba-0c19-4787-9ed8-69b454750f9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:01.658256 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f99e5ba-0c19-4787-9ed8-69b454750f9b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:01.754596 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1449/2878] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:25:01 2022] POST /identity/v3/users => generated 303 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:25:01.758672 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc5c99-704f-4bc4-ab4f-08cc0abc79f8 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:01.769530 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc5c99-704f-4bc4-ab4f-08cc0abc79f8 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:01.774244 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ff313-48dc-401e-8d23-fb134ed78417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:01.774848 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ff313-48dc-401e-8d23-fb134ed78417 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:01.775127 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ff313-48dc-401e-8d23-fb134ed78417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:01.775399 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ff313-48dc-401e-8d23-fb134ed78417 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:01.776043 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ff313-48dc-401e-8d23-fb134ed78417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:01.777641 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ff313-48dc-401e-8d23-fb134ed78417 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:01.788014 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1430/2879] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:01.791673 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99e5ba-0c19-4787-9ed8-69b454750f9b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:01.801365 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99e5ba-0c19-4787-9ed8-69b454750f9b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:01.805828 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba5e2a-19c3-4518-a567-c66126b9945e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:01.806767 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba5e2a-19c3-4518-a567-c66126b9945e admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:01.807048 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba5e2a-19c3-4518-a567-c66126b9945e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:01.807316 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba5e2a-19c3-4518-a567-c66126b9945e admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:01.807955 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba5e2a-19c3-4518-a567-c66126b9945e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:01.809553 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba5e2a-19c3-4518-a567-c66126b9945e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:01.820023 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1450/2880] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:01.823998 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ff313-48dc-401e-8d23-fb134ed78417 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:01.834302 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ff313-48dc-401e-8d23-fb134ed78417 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:01.839091 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0969b3-c9c7-4324-a71d-219411ddd5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:01.839718 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0969b3-c9c7-4324-a71d-219411ddd5eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:01.839997 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0969b3-c9c7-4324-a71d-219411ddd5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:01.840302 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0969b3-c9c7-4324-a71d-219411ddd5eb admin admin] PATH_INFO: `/v3/projects/e82c12b2f62e4624a68b26a1d15921ff/users/7c5aa1dc1ff347e88a06a8430ac36484/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:01.855039 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0969b3-c9c7-4324-a71d-219411ddd5eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e82c12b2f62e4624a68b26a1d15921ff, user_id=7c5aa1dc1ff347e88a06a8430ac36484, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:01.857798 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0969b3-c9c7-4324-a71d-219411ddd5eb admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:01.921060 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1431/2881] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:01 2022] PUT /identity/v3/projects/e82c12b2f62e4624a68b26a1d15921ff/users/7c5aa1dc1ff347e88a06a8430ac36484/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:01.931582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e625535-4b6e-4373-9605-053e35c8dd66 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:01.931864 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e625535-4b6e-4373-9605-053e35c8dd66 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:01.932179 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e625535-4b6e-4373-9605-053e35c8dd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:01.959909 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9e625535-4b6e-4373-9605-053e35c8dd66 None None] MFA Rules not processed for user `7c5aa1dc1ff347e88a06a8430ac36484`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:01.961150 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9e625535-4b6e-4373-9605-053e35c8dd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:02.024374 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1451/2882] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:25:01 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:02.029456 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0969b3-c9c7-4324-a71d-219411ddd5eb admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:02.050721 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:02.091164 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0969b3-c9c7-4324-a71d-219411ddd5eb admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:02.095836 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec476613-9aaf-4d7c-9de0-d9bdcde1b355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:02.096457 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec476613-9aaf-4d7c-9de0-d9bdcde1b355 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:02.097022 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec476613-9aaf-4d7c-9de0-d9bdcde1b355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:02.097022 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec476613-9aaf-4d7c-9de0-d9bdcde1b355 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:02.097623 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec476613-9aaf-4d7c-9de0-d9bdcde1b355 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:02.099519 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec476613-9aaf-4d7c-9de0-d9bdcde1b355 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:02.175354 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1432/2883] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:25:02 2022] POST /identity/v3/projects => generated 405 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:02.179387 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e625535-4b6e-4373-9605-053e35c8dd66 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:02.190486 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e625535-4b6e-4373-9605-053e35c8dd66 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:02.202927 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:02.246323 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb15388-712f-4577-b03e-5fa256c742d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:02.247465 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb15388-712f-4577-b03e-5fa256c742d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:02.247977 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb15388-712f-4577-b03e-5fa256c742d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:02.248455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb15388-712f-4577-b03e-5fa256c742d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:02.249432 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb15388-712f-4577-b03e-5fa256c742d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:02.251392 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb15388-712f-4577-b03e-5fa256c742d9 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:02.317029 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1452/2884] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:25:02 2022] POST /identity/v3/users => generated 309 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:02.320945 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec476613-9aaf-4d7c-9de0-d9bdcde1b355 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:02.331947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec476613-9aaf-4d7c-9de0-d9bdcde1b355 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:02.337102 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfbd95-9785-4588-be90-aa4dbf6163f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:02.337701 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfbd95-9785-4588-be90-aa4dbf6163f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:02.337979 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfbd95-9785-4588-be90-aa4dbf6163f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:02.338251 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfbd95-9785-4588-be90-aa4dbf6163f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:02.338896 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dfbd95-9785-4588-be90-aa4dbf6163f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:02.340547 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dfbd95-9785-4588-be90-aa4dbf6163f8 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:02.365877 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1433/2885] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:02.370092 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb15388-712f-4577-b03e-5fa256c742d9 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:02.380852 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb15388-712f-4577-b03e-5fa256c742d9 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:02.385521 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004b419-9e06-47c4-ad09-d36f6a96c92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:02.386155 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004b419-9e06-47c4-ad09-d36f6a96c92c admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:02.386553 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004b419-9e06-47c4-ad09-d36f6a96c92c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:02.386815 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004b419-9e06-47c4-ad09-d36f6a96c92c admin admin] PATH_INFO: `/v3/projects/5fb24482d8eb4ff89b04d09a93b64d08/users/7ded25656b2e4a8caea44b95232c23af/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:02.403178 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1004b419-9e06-47c4-ad09-d36f6a96c92c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fb24482d8eb4ff89b04d09a93b64d08, user_id=7ded25656b2e4a8caea44b95232c23af, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:02.408325 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1004b419-9e06-47c4-ad09-d36f6a96c92c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:02.506021 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1453/2886] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:02 2022] PUT /identity/v3/projects/5fb24482d8eb4ff89b04d09a93b64d08/users/7ded25656b2e4a8caea44b95232c23af/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:02.510070 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfbd95-9785-4588-be90-aa4dbf6163f8 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:02.519969 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfbd95-9785-4588-be90-aa4dbf6163f8 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:02.581586 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2b84c-de38-427f-8637-fdb8be7a0455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:02.582110 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2b84c-de38-427f-8637-fdb8be7a0455 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:02.582387 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2b84c-de38-427f-8637-fdb8be7a0455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:02.582658 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2b84c-de38-427f-8637-fdb8be7a0455 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:02.583296 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2b84c-de38-427f-8637-fdb8be7a0455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:02.585049 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2b84c-de38-427f-8637-fdb8be7a0455 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:02.595822 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1434/2887] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:02.600120 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004b419-9e06-47c4-ad09-d36f6a96c92c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:02.610960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004b419-9e06-47c4-ad09-d36f6a96c92c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:02.615476 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd8e73-f551-47f8-934c-0e87c9616642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:02.616140 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd8e73-f551-47f8-934c-0e87c9616642 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:02.616420 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd8e73-f551-47f8-934c-0e87c9616642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:02.616689 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd8e73-f551-47f8-934c-0e87c9616642 admin admin] PATH_INFO: `/v3/projects/5fb24482d8eb4ff89b04d09a93b64d08/users/7ded25656b2e4a8caea44b95232c23af/roles/fcbf519fbc84458f913671a5a1d0914c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:02.618856 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd8e73-f551-47f8-934c-0e87c9616642 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fb24482d8eb4ff89b04d09a93b64d08, user_id=7ded25656b2e4a8caea44b95232c23af, role_id=fcbf519fbc84458f913671a5a1d0914c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:02.621525 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd8e73-f551-47f8-934c-0e87c9616642 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:02.631870 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1454/2888] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:02 2022] PUT /identity/v3/projects/5fb24482d8eb4ff89b04d09a93b64d08/users/7ded25656b2e4a8caea44b95232c23af/roles/fcbf519fbc84458f913671a5a1d0914c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:02.642070 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da21877-8a1c-4fce-b2c5-e718309d2033 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:02.642362 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da21877-8a1c-4fce-b2c5-e718309d2033 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:02.642636 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da21877-8a1c-4fce-b2c5-e718309d2033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:02.672459 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-3da21877-8a1c-4fce-b2c5-e718309d2033 None None] MFA Rules not processed for user `7ded25656b2e4a8caea44b95232c23af`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:02.673664 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3da21877-8a1c-4fce-b2c5-e718309d2033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:02.747945 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1435/2889] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:25:02 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:02.758639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd8e73-f551-47f8-934c-0e87c9616642 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:02.760376 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c2fd8e73-f551-47f8-934c-0e87c9616642 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:02.837000 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd8e73-f551-47f8-934c-0e87c9616642 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:02.841112 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e4630-f088-4485-9bef-21db6c436d6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:02.841707 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e4630-f088-4485-9bef-21db6c436d6e service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:02.841991 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e4630-f088-4485-9bef-21db6c436d6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:02.842268 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e4630-f088-4485-9bef-21db6c436d6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:02.851112 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e4630-f088-4485-9bef-21db6c436d6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:02.852577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e4630-f088-4485-9bef-21db6c436d6e service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:02.862522 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1455/2890] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:02 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:03.007677 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da21877-8a1c-4fce-b2c5-e718309d2033 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:03.024639 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:03.070103 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da21877-8a1c-4fce-b2c5-e718309d2033 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:03.074638 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d072be2-1a78-4e93-a48f-e1d274c20baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:03.075260 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d072be2-1a78-4e93-a48f-e1d274c20baf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:03.075535 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d072be2-1a78-4e93-a48f-e1d274c20baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:03.075805 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d072be2-1a78-4e93-a48f-e1d274c20baf admin admin] PATH_INFO: `/v3/users/7c5aa1dc1ff347e88a06a8430ac36484` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:03.076947 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d072be2-1a78-4e93-a48f-e1d274c20baf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c5aa1dc1ff347e88a06a8430ac36484)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:03.078858 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d072be2-1a78-4e93-a48f-e1d274c20baf admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:03.349364 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1d072be2-1a78-4e93-a48f-e1d274c20baf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c5aa1dc1ff347e88a06a8430ac36484'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:03.350181 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1d072be2-1a78-4e93-a48f-e1d274c20baf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c5aa1dc1ff347e88a06a8430ac36484'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:03.356865 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1d072be2-1a78-4e93-a48f-e1d274c20baf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c5aa1dc1ff347e88a06a8430ac36484'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:03.368884 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1d072be2-1a78-4e93-a48f-e1d274c20baf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c5aa1dc1ff347e88a06a8430ac36484'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:03.369646 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1d072be2-1a78-4e93-a48f-e1d274c20baf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c5aa1dc1ff347e88a06a8430ac36484'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:03.442253 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1436/2891] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:25:03 2022] DELETE /identity/v3/users/7c5aa1dc1ff347e88a06a8430ac36484 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:03.446761 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f40ced-40a6-40d9-b754-12dc8fc21844 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:03.447278 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f40ced-40a6-40d9-b754-12dc8fc21844 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:03.447433 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f40ced-40a6-40d9-b754-12dc8fc21844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:03.467480 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e7f40ced-40a6-40d9-b754-12dc8fc21844 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:03.468777 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e7f40ced-40a6-40d9-b754-12dc8fc21844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:03.514878 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:03.526690 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1456/2892] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:03.536969 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d072be2-1a78-4e93-a48f-e1d274c20baf admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:03.538361 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-1d072be2-1a78-4e93-a48f-e1d274c20baf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:03.595082 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d072be2-1a78-4e93-a48f-e1d274c20baf admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:03.598619 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a5151-7f2f-4e05-83a7-bd598ea0face service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:03.603016 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a5151-7f2f-4e05-83a7-bd598ea0face service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:03.603016 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a5151-7f2f-4e05-83a7-bd598ea0face service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:03.603016 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a5151-7f2f-4e05-83a7-bd598ea0face service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:03.608654 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9a5151-7f2f-4e05-83a7-bd598ea0face service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:03.610063 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9a5151-7f2f-4e05-83a7-bd598ea0face service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:03.619268 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1437/2893] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:04.070697 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f40ced-40a6-40d9-b754-12dc8fc21844 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:04.072159 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e7f40ced-40a6-40d9-b754-12dc8fc21844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:04.085496 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f40ced-40a6-40d9-b754-12dc8fc21844 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:04.090490 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60bcf8-0ad4-44d2-a845-387bd66d1310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:04.091145 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60bcf8-0ad4-44d2-a845-387bd66d1310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:04.091427 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60bcf8-0ad4-44d2-a845-387bd66d1310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:04.091698 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60bcf8-0ad4-44d2-a845-387bd66d1310 admin admin] PATH_INFO: `/v3/projects/e82c12b2f62e4624a68b26a1d15921ff` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:04.092846 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee60bcf8-0ad4-44d2-a845-387bd66d1310 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e82c12b2f62e4624a68b26a1d15921ff)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:04.094770 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee60bcf8-0ad4-44d2-a845-387bd66d1310 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:04.206074 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ee60bcf8-0ad4-44d2-a845-387bd66d1310 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e82c12b2f62e4624a68b26a1d15921ff', 'request_id': 'req-ee60bcf8-0ad4-44d2-a845-387bd66d1310'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:04.212661 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ee60bcf8-0ad4-44d2-a845-387bd66d1310 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e82c12b2f62e4624a68b26a1d15921ff', 'request_id': 'req-ee60bcf8-0ad4-44d2-a845-387bd66d1310'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:04.274619 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ee60bcf8-0ad4-44d2-a845-387bd66d1310 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e82c12b2f62e4624a68b26a1d15921ff', 'request_id': 'req-ee60bcf8-0ad4-44d2-a845-387bd66d1310'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:04.285068 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ee60bcf8-0ad4-44d2-a845-387bd66d1310 admin admin] The token cache is being invalidate because project e82c12b2f62e4624a68b26a1d15921ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:25:04.291834 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ee60bcf8-0ad4-44d2-a845-387bd66d1310 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:04.291834 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ee60bcf8-0ad4-44d2-a845-387bd66d1310 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:04.291834 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1457/2894] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:25:04 2022] DELETE /identity/v3/projects/e82c12b2f62e4624a68b26a1d15921ff => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:04.292223 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a5151-7f2f-4e05-83a7-bd598ea0face service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:04.301706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a5151-7f2f-4e05-83a7-bd598ea0face service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:04.334296 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:04.358109 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b9be63-51df-496f-be72-9f098e4dd7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:04.358798 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9be63-51df-496f-be72-9f098e4dd7a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:04.359012 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9be63-51df-496f-be72-9f098e4dd7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:04.359286 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9be63-51df-496f-be72-9f098e4dd7a3 admin admin] PATH_INFO: `/v3/users/7ded25656b2e4a8caea44b95232c23af` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:04.360446 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b9be63-51df-496f-be72-9f098e4dd7a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ded25656b2e4a8caea44b95232c23af)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:04.362373 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b9be63-51df-496f-be72-9f098e4dd7a3 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:04.558830 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a4b9be63-51df-496f-be72-9f098e4dd7a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ded25656b2e4a8caea44b95232c23af'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:04.559422 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a4b9be63-51df-496f-be72-9f098e4dd7a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ded25656b2e4a8caea44b95232c23af'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:04.566392 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a4b9be63-51df-496f-be72-9f098e4dd7a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ded25656b2e4a8caea44b95232c23af'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:04.578434 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a4b9be63-51df-496f-be72-9f098e4dd7a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ded25656b2e4a8caea44b95232c23af'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:04.579074 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a4b9be63-51df-496f-be72-9f098e4dd7a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ded25656b2e4a8caea44b95232c23af'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:04.657719 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1438/2895] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:25:04 2022] DELETE /identity/v3/users/7ded25656b2e4a8caea44b95232c23af => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:05.077080 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60bcf8-0ad4-44d2-a845-387bd66d1310 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.078537 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ee60bcf8-0ad4-44d2-a845-387bd66d1310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:05.092850 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60bcf8-0ad4-44d2-a845-387bd66d1310 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.153561 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f378cc-dc58-4853-bfae-f4603bd9f855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.154196 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f378cc-dc58-4853-bfae-f4603bd9f855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.154476 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f378cc-dc58-4853-bfae-f4603bd9f855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.154771 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f378cc-dc58-4853-bfae-f4603bd9f855 admin admin] PATH_INFO: `/v3/projects/5fb24482d8eb4ff89b04d09a93b64d08` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.155933 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f378cc-dc58-4853-bfae-f4603bd9f855 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fb24482d8eb4ff89b04d09a93b64d08)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.157874 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f378cc-dc58-4853-bfae-f4603bd9f855 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.266179 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a6f378cc-dc58-4853-bfae-f4603bd9f855 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fb24482d8eb4ff89b04d09a93b64d08', 'request_id': 'req-a6f378cc-dc58-4853-bfae-f4603bd9f855'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:05.272342 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a6f378cc-dc58-4853-bfae-f4603bd9f855 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fb24482d8eb4ff89b04d09a93b64d08', 'request_id': 'req-a6f378cc-dc58-4853-bfae-f4603bd9f855'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:05.343477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a6f378cc-dc58-4853-bfae-f4603bd9f855 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fb24482d8eb4ff89b04d09a93b64d08', 'request_id': 'req-a6f378cc-dc58-4853-bfae-f4603bd9f855'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:05.362143 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a6f378cc-dc58-4853-bfae-f4603bd9f855 admin admin] The token cache is being invalidate because project 5fb24482d8eb4ff89b04d09a93b64d08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:25:05.362435 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a6f378cc-dc58-4853-bfae-f4603bd9f855 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:05.363126 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a6f378cc-dc58-4853-bfae-f4603bd9f855 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:05.364725 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1458/2896] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:25:05 2022] DELETE /identity/v3/projects/5fb24482d8eb4ff89b04d09a93b64d08 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:05.372483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cd2f3-69c6-4f49-92b7-184aad30f555 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.373040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cd2f3-69c6-4f49-92b7-184aad30f555 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.373237 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cd2f3-69c6-4f49-92b7-184aad30f555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.394701 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-149cd2f3-69c6-4f49-92b7-184aad30f555 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:05.395951 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-149cd2f3-69c6-4f49-92b7-184aad30f555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:05.453851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1439/2897] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:05.515151 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858a6a2-b7fe-490b-8ae2-bda8acc83d9e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.515541 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858a6a2-b7fe-490b-8ae2-bda8acc83d9e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.515883 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858a6a2-b7fe-490b-8ae2-bda8acc83d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.530240 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:05.535718 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1858a6a2-b7fe-490b-8ae2-bda8acc83d9e None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:05.536941 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1858a6a2-b7fe-490b-8ae2-bda8acc83d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:05.545163 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1459/2898] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:05.549459 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149cd2f3-69c6-4f49-92b7-184aad30f555 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.559906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149cd2f3-69c6-4f49-92b7-184aad30f555 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.564463 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8007b4-aa05-4552-9445-0db7aa09b402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.569942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8007b4-aa05-4552-9445-0db7aa09b402 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.569942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8007b4-aa05-4552-9445-0db7aa09b402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.569942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8007b4-aa05-4552-9445-0db7aa09b402 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.569942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8007b4-aa05-4552-9445-0db7aa09b402 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:25:05.569942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8007b4-aa05-4552-9445-0db7aa09b402 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.569942 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8007b4-aa05-4552-9445-0db7aa09b402 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.579627 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1440/2899] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:25:05.583784 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858a6a2-b7fe-490b-8ae2-bda8acc83d9e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.593386 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858a6a2-b7fe-490b-8ae2-bda8acc83d9e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.598657 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54897b9a-e0a2-4860-af22-8bdedcc0bd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.598815 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54897b9a-e0a2-4860-af22-8bdedcc0bd96 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.599049 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54897b9a-e0a2-4860-af22-8bdedcc0bd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.599317 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54897b9a-e0a2-4860-af22-8bdedcc0bd96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.599950 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54897b9a-e0a2-4860-af22-8bdedcc0bd96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.601867 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54897b9a-e0a2-4860-af22-8bdedcc0bd96 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.706242 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1460/2900] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:25:05 2022] POST /identity/v3/projects => generated 367 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:05.710789 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8007b4-aa05-4552-9445-0db7aa09b402 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.721367 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8007b4-aa05-4552-9445-0db7aa09b402 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.762540 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:05.776592 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e813f16-a184-4e7b-bb9e-2c63e289e168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.777186 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e813f16-a184-4e7b-bb9e-2c63e289e168 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.777465 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e813f16-a184-4e7b-bb9e-2c63e289e168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.777735 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e813f16-a184-4e7b-bb9e-2c63e289e168 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.778369 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e813f16-a184-4e7b-bb9e-2c63e289e168 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.780282 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e813f16-a184-4e7b-bb9e-2c63e289e168 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.874866 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1441/2901] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:25:05 2022] POST /identity/v3/users => generated 284 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:05.878865 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54897b9a-e0a2-4860-af22-8bdedcc0bd96 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.888993 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54897b9a-e0a2-4860-af22-8bdedcc0bd96 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.893591 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d427a1f-9e10-4bbe-93fc-5e8403507424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.894171 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d427a1f-9e10-4bbe-93fc-5e8403507424 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.894446 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d427a1f-9e10-4bbe-93fc-5e8403507424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.894712 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d427a1f-9e10-4bbe-93fc-5e8403507424 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.895367 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d427a1f-9e10-4bbe-93fc-5e8403507424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.896987 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d427a1f-9e10-4bbe-93fc-5e8403507424 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.907300 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1461/2902] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:05.911991 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e813f16-a184-4e7b-bb9e-2c63e289e168 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.922673 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e813f16-a184-4e7b-bb9e-2c63e289e168 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.927854 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f957781f-3ea4-4187-bcc3-a7972135ccf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.928573 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957781f-3ea4-4187-bcc3-a7972135ccf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.928859 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957781f-3ea4-4187-bcc3-a7972135ccf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.929132 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957781f-3ea4-4187-bcc3-a7972135ccf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.929882 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f957781f-3ea4-4187-bcc3-a7972135ccf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.931465 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f957781f-3ea4-4187-bcc3-a7972135ccf5 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.942986 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1442/2903] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:05.947181 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d427a1f-9e10-4bbe-93fc-5e8403507424 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.957113 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d427a1f-9e10-4bbe-93fc-5e8403507424 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.961664 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a136b39-577b-4cc2-a3d4-dc9cd326866a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.962566 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a136b39-577b-4cc2-a3d4-dc9cd326866a admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.962840 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a136b39-577b-4cc2-a3d4-dc9cd326866a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.963109 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a136b39-577b-4cc2-a3d4-dc9cd326866a admin admin] PATH_INFO: `/v3/projects/3c6e718abe3844b4b57004d88e7a9b7d/users/ec5601cc33344f2fbde7cba05a81a22d/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.977969 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a136b39-577b-4cc2-a3d4-dc9cd326866a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c6e718abe3844b4b57004d88e7a9b7d, user_id=ec5601cc33344f2fbde7cba05a81a22d, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.980735 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a136b39-577b-4cc2-a3d4-dc9cd326866a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:06.093123 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1462/2904] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:05 2022] PUT /identity/v3/projects/3c6e718abe3844b4b57004d88e7a9b7d/users/ec5601cc33344f2fbde7cba05a81a22d/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:06.103046 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0ca6e-382b-496d-8586-ac412b199756 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:06.103334 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0ca6e-382b-496d-8586-ac412b199756 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:06.103608 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0ca6e-382b-496d-8586-ac412b199756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:06.132172 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-5ad0ca6e-382b-496d-8586-ac412b199756 None None] MFA Rules not processed for user `ec5601cc33344f2fbde7cba05a81a22d`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:06.133319 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5ad0ca6e-382b-496d-8586-ac412b199756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:06.195780 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1443/2905] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:25:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:06.207966 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a136b39-577b-4cc2-a3d4-dc9cd326866a admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:06.209714 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9a136b39-577b-4cc2-a3d4-dc9cd326866a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:06.285829 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a136b39-577b-4cc2-a3d4-dc9cd326866a admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:06.289464 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b54b4-37cb-464d-83a3-e36b4dfd2aed service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7832e97d7e97439e821268d05816ae3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:06.290062 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b54b4-37cb-464d-83a3-e36b4dfd2aed service glance] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:06.290336 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b54b4-37cb-464d-83a3-e36b4dfd2aed service glance] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:06.290607 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b54b4-37cb-464d-83a3-e36b4dfd2aed service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:06.299096 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b54b4-37cb-464d-83a3-e36b4dfd2aed service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:06.300579 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b54b4-37cb-464d-83a3-e36b4dfd2aed service glance] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:06.311992 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1463/2906] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:16.192915 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=46,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:16.307026 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=59,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:51.056793 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0ca6e-382b-496d-8586-ac412b199756 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:51.118169 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0ca6e-382b-496d-8586-ac412b199756 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:51.122668 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab726a1e-0a87-49c7-9af9-d97135a744e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:51.123287 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab726a1e-0a87-49c7-9af9-d97135a744e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:51.123569 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab726a1e-0a87-49c7-9af9-d97135a744e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:51.123843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab726a1e-0a87-49c7-9af9-d97135a744e3 admin admin] PATH_INFO: `/v3/users/ec5601cc33344f2fbde7cba05a81a22d` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:51.124991 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab726a1e-0a87-49c7-9af9-d97135a744e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec5601cc33344f2fbde7cba05a81a22d)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:51.127702 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab726a1e-0a87-49c7-9af9-d97135a744e3 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:51.311120 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ab726a1e-0a87-49c7-9af9-d97135a744e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec5601cc33344f2fbde7cba05a81a22d'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:51.311764 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ab726a1e-0a87-49c7-9af9-d97135a744e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec5601cc33344f2fbde7cba05a81a22d'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:51.319038 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ab726a1e-0a87-49c7-9af9-d97135a744e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec5601cc33344f2fbde7cba05a81a22d'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:51.330738 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ab726a1e-0a87-49c7-9af9-d97135a744e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec5601cc33344f2fbde7cba05a81a22d'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:51.331417 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ab726a1e-0a87-49c7-9af9-d97135a744e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec5601cc33344f2fbde7cba05a81a22d'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:51.417460 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1444/2907] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:25:51 2022] DELETE /identity/v3/users/ec5601cc33344f2fbde7cba05a81a22d => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:51.422460 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a4571-77d8-4d91-bed5-db2e3d5f4a76 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:51.422943 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a4571-77d8-4d91-bed5-db2e3d5f4a76 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:51.423113 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a4571-77d8-4d91-bed5-db2e3d5f4a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:51.443264 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-3f3a4571-77d8-4d91-bed5-db2e3d5f4a76 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:51.444494 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3f3a4571-77d8-4d91-bed5-db2e3d5f4a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:51.502582 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1464/2908] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:25:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:51.512620 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab726a1e-0a87-49c7-9af9-d97135a744e3 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:51.514032 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ab726a1e-0a87-49c7-9af9-d97135a744e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:51.570962 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab726a1e-0a87-49c7-9af9-d97135a744e3 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:51.574590 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39f418-f652-4f9b-80dc-fe4918c6f270 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:51.575194 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39f418-f652-4f9b-80dc-fe4918c6f270 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:51.575473 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39f418-f652-4f9b-80dc-fe4918c6f270 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:51.575741 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39f418-f652-4f9b-80dc-fe4918c6f270 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:51.585146 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f39f418-f652-4f9b-80dc-fe4918c6f270 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:51.586570 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f39f418-f652-4f9b-80dc-fe4918c6f270 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:51.596757 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1445/2909] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:51.973341 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a4571-77d8-4d91-bed5-db2e3d5f4a76 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:51.974804 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3f3a4571-77d8-4d91-bed5-db2e3d5f4a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:51.988102 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a4571-77d8-4d91-bed5-db2e3d5f4a76 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:51.992695 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d68301-e156-4b95-883d-81f72d885279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:51.993333 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d68301-e156-4b95-883d-81f72d885279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:51.993625 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d68301-e156-4b95-883d-81f72d885279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:51.993904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d68301-e156-4b95-883d-81f72d885279 admin admin] PATH_INFO: `/v3/projects/3c6e718abe3844b4b57004d88e7a9b7d` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:51.995073 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d68301-e156-4b95-883d-81f72d885279 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c6e718abe3844b4b57004d88e7a9b7d)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:51.997057 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d68301-e156-4b95-883d-81f72d885279 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:52.104316 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-80d68301-e156-4b95-883d-81f72d885279 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c6e718abe3844b4b57004d88e7a9b7d', 'request_id': 'req-80d68301-e156-4b95-883d-81f72d885279'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:52.110664 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-80d68301-e156-4b95-883d-81f72d885279 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c6e718abe3844b4b57004d88e7a9b7d', 'request_id': 'req-80d68301-e156-4b95-883d-81f72d885279'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:52.196386 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-80d68301-e156-4b95-883d-81f72d885279 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c6e718abe3844b4b57004d88e7a9b7d', 'request_id': 'req-80d68301-e156-4b95-883d-81f72d885279'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:52.207618 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-80d68301-e156-4b95-883d-81f72d885279 admin admin] The token cache is being invalidate because project 3c6e718abe3844b4b57004d88e7a9b7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:25:52.213353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-80d68301-e156-4b95-883d-81f72d885279 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:52.213353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-80d68301-e156-4b95-883d-81f72d885279 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:52.213353 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1465/2910] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:25:51 2022] DELETE /identity/v3/projects/3c6e718abe3844b4b57004d88e7a9b7d => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:52.215230 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8442b2-a8ae-43bd-ad19-1b7ae0b2d62c None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:52.215678 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8442b2-a8ae-43bd-ad19-1b7ae0b2d62c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:52.215788 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8442b2-a8ae-43bd-ad19-1b7ae0b2d62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:52.235040 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-9f8442b2-a8ae-43bd-ad19-1b7ae0b2d62c None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:52.241420 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-9f8442b2-a8ae-43bd-ad19-1b7ae0b2d62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:52.253829 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:52.294586 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1446/2911] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:25:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:52.304215 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7625a01-889d-4a7d-8419-08733420c325 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:52.304499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7625a01-889d-4a7d-8419-08733420c325 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:52.304770 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7625a01-889d-4a7d-8419-08733420c325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:52.334732 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e7625a01-889d-4a7d-8419-08733420c325 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:52.335961 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e7625a01-889d-4a7d-8419-08733420c325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:52.344494 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1466/2912] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:25:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:52.348859 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8442b2-a8ae-43bd-ad19-1b7ae0b2d62c None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:52.359704 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8442b2-a8ae-43bd-ad19-1b7ae0b2d62c None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:52.366777 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab98d1-7045-4b8d-8e2b-a970632f8527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:52.367762 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab98d1-7045-4b8d-8e2b-a970632f8527 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:52.368318 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab98d1-7045-4b8d-8e2b-a970632f8527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:52.368799 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab98d1-7045-4b8d-8e2b-a970632f8527 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:52.369633 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab98d1-7045-4b8d-8e2b-a970632f8527 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:25:52.369929 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab98d1-7045-4b8d-8e2b-a970632f8527 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:52.371508 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab98d1-7045-4b8d-8e2b-a970632f8527 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:52.384120 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1447/2913] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:25:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:25:52.388839 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7625a01-889d-4a7d-8419-08733420c325 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:52.399214 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7625a01-889d-4a7d-8419-08733420c325 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:52.403925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2386c13-bf28-4256-ae45-d82a13cac173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:52.404553 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2386c13-bf28-4256-ae45-d82a13cac173 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:52.404832 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2386c13-bf28-4256-ae45-d82a13cac173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:52.405102 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2386c13-bf28-4256-ae45-d82a13cac173 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:52.405781 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2386c13-bf28-4256-ae45-d82a13cac173 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:52.407727 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2386c13-bf28-4256-ae45-d82a13cac173 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:52.506639 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1467/2914] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:25:52 2022] POST /identity/v3/projects => generated 375 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:52.510936 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab98d1-7045-4b8d-8e2b-a970632f8527 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:52.520973 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab98d1-7045-4b8d-8e2b-a970632f8527 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:52.577619 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35397556-3ce9-44f9-b7bb-2c542fae9378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:52.578220 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35397556-3ce9-44f9-b7bb-2c542fae9378 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:52.578499 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35397556-3ce9-44f9-b7bb-2c542fae9378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:52.578775 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35397556-3ce9-44f9-b7bb-2c542fae9378 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:52.579400 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35397556-3ce9-44f9-b7bb-2c542fae9378 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:52.581383 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35397556-3ce9-44f9-b7bb-2c542fae9378 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:52.676394 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1448/2915] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:25:52 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:52.679794 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2386c13-bf28-4256-ae45-d82a13cac173 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:52.692670 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2386c13-bf28-4256-ae45-d82a13cac173 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:52.697485 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6369315-5389-46fb-83d3-4dd42692be63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:52.698091 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6369315-5389-46fb-83d3-4dd42692be63 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:52.698374 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6369315-5389-46fb-83d3-4dd42692be63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:52.698650 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6369315-5389-46fb-83d3-4dd42692be63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:52.699296 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6369315-5389-46fb-83d3-4dd42692be63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:52.700940 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6369315-5389-46fb-83d3-4dd42692be63 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:52.711184 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1468/2916] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:52.715409 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35397556-3ce9-44f9-b7bb-2c542fae9378 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:52.725984 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35397556-3ce9-44f9-b7bb-2c542fae9378 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:52.730690 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80314c83-d5ac-4763-b4d8-d0c15ac57c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:52.731280 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80314c83-d5ac-4763-b4d8-d0c15ac57c0a admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:52.731562 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80314c83-d5ac-4763-b4d8-d0c15ac57c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:52.731838 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80314c83-d5ac-4763-b4d8-d0c15ac57c0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:52.732522 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80314c83-d5ac-4763-b4d8-d0c15ac57c0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:52.734111 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80314c83-d5ac-4763-b4d8-d0c15ac57c0a admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:52.745551 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1449/2917] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:52.749524 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6369315-5389-46fb-83d3-4dd42692be63 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:52.759322 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6369315-5389-46fb-83d3-4dd42692be63 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:52.763936 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91600e21-2b1c-4177-9046-30663f942c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:52.764614 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91600e21-2b1c-4177-9046-30663f942c39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:52.764897 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91600e21-2b1c-4177-9046-30663f942c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:52.765172 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91600e21-2b1c-4177-9046-30663f942c39 admin admin] PATH_INFO: `/v3/projects/ccdaeb5238614decb2020f32a52c0b20/users/e1d18999210f4a7d8ef45645e7781700/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:52.780601 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91600e21-2b1c-4177-9046-30663f942c39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccdaeb5238614decb2020f32a52c0b20, user_id=e1d18999210f4a7d8ef45645e7781700, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:52.783240 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91600e21-2b1c-4177-9046-30663f942c39 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:52.879862 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1469/2918] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:52 2022] PUT /identity/v3/projects/ccdaeb5238614decb2020f32a52c0b20/users/e1d18999210f4a7d8ef45645e7781700/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:52.889997 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03269e6-3bcd-47bd-98e0-50f8efcda067 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:52.890281 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03269e6-3bcd-47bd-98e0-50f8efcda067 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:52.890565 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03269e6-3bcd-47bd-98e0-50f8efcda067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:52.918196 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-a03269e6-3bcd-47bd-98e0-50f8efcda067 None None] MFA Rules not processed for user `e1d18999210f4a7d8ef45645e7781700`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:52.919420 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a03269e6-3bcd-47bd-98e0-50f8efcda067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:52.984409 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1450/2919] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:25:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:52.994688 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91600e21-2b1c-4177-9046-30663f942c39 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:52.997398 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-91600e21-2b1c-4177-9046-30663f942c39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:53.011452 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:53.055461 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91600e21-2b1c-4177-9046-30663f942c39 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:53.059223 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0326ce6d-3240-485e-bf6a-9558a97b2e68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:53.061455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0326ce6d-3240-485e-bf6a-9558a97b2e68 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:53.061455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0326ce6d-3240-485e-bf6a-9558a97b2e68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:53.061455 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0326ce6d-3240-485e-bf6a-9558a97b2e68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:53.068908 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0326ce6d-3240-485e-bf6a-9558a97b2e68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:53.070332 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0326ce6d-3240-485e-bf6a-9558a97b2e68 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:53.079230 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1470/2920] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:25:54.810563 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93a2eb-ca2d-439a-b506-09fcebfd61e4 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:54.811094 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93a2eb-ca2d-439a-b506-09fcebfd61e4 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:54.811094 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93a2eb-ca2d-439a-b506-09fcebfd61e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:54.831611 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-8f93a2eb-ca2d-439a-b506-09fcebfd61e4 None None] MFA Rules not processed for user `e1d18999210f4a7d8ef45645e7781700`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:54.832949 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8f93a2eb-ca2d-439a-b506-09fcebfd61e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:54.840105 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1451/2921] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:25:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:54.874286 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0326ce6d-3240-485e-bf6a-9558a97b2e68 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:54.878326 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0326ce6d-3240-485e-bf6a-9558a97b2e68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:54.932257 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0326ce6d-3240-485e-bf6a-9558a97b2e68 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:54.935814 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44691c71-ce7f-462f-99f5-edaaed542abf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7832e97d7e97439e821268d05816ae3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:54.936436 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44691c71-ce7f-462f-99f5-edaaed542abf service glance] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:54.936717 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44691c71-ce7f-462f-99f5-edaaed542abf service glance] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:54.936987 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44691c71-ce7f-462f-99f5-edaaed542abf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:54.938709 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-44691c71-ce7f-462f-99f5-edaaed542abf service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:55.028922 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44691c71-ce7f-462f-99f5-edaaed542abf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:55.030301 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44691c71-ce7f-462f-99f5-edaaed542abf service glance] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:55.050186 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1471/2922] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:56.591765 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93a2eb-ca2d-439a-b506-09fcebfd61e4 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:56.593196 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8f93a2eb-ca2d-439a-b506-09fcebfd61e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:56.597052 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44691c71-ce7f-462f-99f5-edaaed542abf service glance] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:56.598693 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-44691c71-ce7f-462f-99f5-edaaed542abf service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:56.610642 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:56.640645 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:56.657863 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44691c71-ce7f-462f-99f5-edaaed542abf service glance] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:56.661512 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d73b23-4209-4abb-b764-d63867037c0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:56.662117 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d73b23-4209-4abb-b764-d63867037c0b service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:56.662397 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d73b23-4209-4abb-b764-d63867037c0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:56.662702 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d73b23-4209-4abb-b764-d63867037c0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:56.664461 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-45d73b23-4209-4abb-b764-d63867037c0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:56.670988 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93a2eb-ca2d-439a-b506-09fcebfd61e4 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:56.676932 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a562a20-a8ad-4d65-9384-4a96c191c4f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4f6de2065d4fbea3fa141da760f708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:56.677551 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a562a20-a8ad-4d65-9384-4a96c191c4f3 service placement] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:56.677821 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a562a20-a8ad-4d65-9384-4a96c191c4f3 service placement] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:56.678088 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a562a20-a8ad-4d65-9384-4a96c191c4f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:56.678901 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d73b23-4209-4abb-b764-d63867037c0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:56.682608 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-6a562a20-a8ad-4d65-9384-4a96c191c4f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:56.682810 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d73b23-4209-4abb-b764-d63867037c0b service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:56.692334 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1472/2923] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:25:56.693439 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a562a20-a8ad-4d65-9384-4a96c191c4f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:56.695015 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a562a20-a8ad-4d65-9384-4a96c191c4f3 service placement] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:56.706156 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1452/2924] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:26:06.317904 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d73b23-4209-4abb-b764-d63867037c0b service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:06.319343 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-45d73b23-4209-4abb-b764-d63867037c0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:06.333864 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d73b23-4209-4abb-b764-d63867037c0b service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:06.339739 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdff2eb-61f3-4263-9770-c899ed35932a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613b845a9b4d432daf8d478fc3970708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:06.340350 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdff2eb-61f3-4263-9770-c899ed35932a service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:06.340625 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdff2eb-61f3-4263-9770-c899ed35932a service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:06.340890 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdff2eb-61f3-4263-9770-c899ed35932a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:06.342523 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-abdff2eb-61f3-4263-9770-c899ed35932a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:06.354816 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdff2eb-61f3-4263-9770-c899ed35932a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:06.356407 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdff2eb-61f3-4263-9770-c899ed35932a service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:06.367130 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1473/2925] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:26:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:26:06.701451 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=27 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:13.622703 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd1c56-3885-4ed4-a7aa-61fb2cad33aa None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:13.623338 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd1c56-3885-4ed4-a7aa-61fb2cad33aa None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:13.623338 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd1c56-3885-4ed4-a7aa-61fb2cad33aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:13.681682 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-7bbd1c56-3885-4ed4-a7aa-61fb2cad33aa None None] MFA Rules not processed for user `8c21ea7d3cbb494886e60a4bd4f39145`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:13.682895 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-7bbd1c56-3885-4ed4-a7aa-61fb2cad33aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:13.746720 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1453/2926] 10.222.0.48 () {58 vars in 1009 bytes} [Fri Sep 9 21:26:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:13.862577 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdff2eb-61f3-4263-9770-c899ed35932a service nova] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:13.863923 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-abdff2eb-61f3-4263-9770-c899ed35932a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:13.952925 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdff2eb-61f3-4263-9770-c899ed35932a service nova] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:13.957849 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e6a3b-7426-485e-a194-8db012ae2772 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d28fa5d49d44120bd5eaec2d4e5fdbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:13.958441 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e6a3b-7426-485e-a194-8db012ae2772 service swift] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:13.958716 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e6a3b-7426-485e-a194-8db012ae2772 service swift] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:13.958984 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e6a3b-7426-485e-a194-8db012ae2772 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:13.967603 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e6a3b-7426-485e-a194-8db012ae2772 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:13.969196 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e6a3b-7426-485e-a194-8db012ae2772 service swift] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:13.977277 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1474/2927] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Sep 9 21:26:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 21:26:14.725982 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527bff8-82bd-480b-8ec7-51f9e6deed32 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:14.726494 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527bff8-82bd-480b-8ec7-51f9e6deed32 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:14.726613 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527bff8-82bd-480b-8ec7-51f9e6deed32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:14.745838 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-2527bff8-82bd-480b-8ec7-51f9e6deed32 None None] MFA Rules not processed for user `8c21ea7d3cbb494886e60a4bd4f39145`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:14.747298 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2527bff8-82bd-480b-8ec7-51f9e6deed32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:14.754662 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1454/2928] 10.222.0.48 () {58 vars in 967 bytes} [Fri Sep 9 21:26:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:14.766830 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e6a3b-7426-485e-a194-8db012ae2772 service swift] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:14.778402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e6a3b-7426-485e-a194-8db012ae2772 service swift] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:14.783378 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b176e274-c060-48a9-9f0f-fecd19a68d3d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d28fa5d49d44120bd5eaec2d4e5fdbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:14.783966 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176e274-c060-48a9-9f0f-fecd19a68d3d service swift] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:14.784278 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176e274-c060-48a9-9f0f-fecd19a68d3d service swift] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:14.784556 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176e274-c060-48a9-9f0f-fecd19a68d3d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:14.793814 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b176e274-c060-48a9-9f0f-fecd19a68d3d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:14.795518 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b176e274-c060-48a9-9f0f-fecd19a68d3d service swift] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:14.803443 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1475/2929] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Sep 9 21:26:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 21:26:15.359459 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021998cf-1d94-49d5-8b45-38844d7f3e84 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:15.359746 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021998cf-1d94-49d5-8b45-38844d7f3e84 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:15.362307 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021998cf-1d94-49d5-8b45-38844d7f3e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:15.380527 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-021998cf-1d94-49d5-8b45-38844d7f3e84 None None] MFA Rules not processed for user `8c21ea7d3cbb494886e60a4bd4f39145`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:15.382001 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-021998cf-1d94-49d5-8b45-38844d7f3e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:15.389520 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1455/2930] 10.222.0.48 () {58 vars in 1009 bytes} [Fri Sep 9 21:26:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:15.401287 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b176e274-c060-48a9-9f0f-fecd19a68d3d service swift] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:15.411869 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b176e274-c060-48a9-9f0f-fecd19a68d3d service swift] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:15.416786 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9028c2a5-bc25-4e93-ae77-eb08e4987ebf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d28fa5d49d44120bd5eaec2d4e5fdbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:15.417365 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028c2a5-bc25-4e93-ae77-eb08e4987ebf service swift] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:15.417641 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028c2a5-bc25-4e93-ae77-eb08e4987ebf service swift] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:15.417936 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028c2a5-bc25-4e93-ae77-eb08e4987ebf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:15.425719 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9028c2a5-bc25-4e93-ae77-eb08e4987ebf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:15.427805 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9028c2a5-bc25-4e93-ae77-eb08e4987ebf service swift] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:15.435751 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1476/2931] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Sep 9 21:26:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 21:26:15.967369 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbd196-e620-49e5-b3c8-24d050d2b69a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:15.967878 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbd196-e620-49e5-b3c8-24d050d2b69a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:15.967989 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbd196-e620-49e5-b3c8-24d050d2b69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:15.987704 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-33bbd196-e620-49e5-b3c8-24d050d2b69a None None] MFA Rules not processed for user `8c21ea7d3cbb494886e60a4bd4f39145`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:15.988950 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-33bbd196-e620-49e5-b3c8-24d050d2b69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:15.996620 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1456/2932] 10.222.0.48 () {58 vars in 967 bytes} [Fri Sep 9 21:26:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:16.008712 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9028c2a5-bc25-4e93-ae77-eb08e4987ebf service swift] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:16.019291 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9028c2a5-bc25-4e93-ae77-eb08e4987ebf service swift] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:16.024342 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239a291-fc5d-490e-a8e3-acaddae71b46 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d28fa5d49d44120bd5eaec2d4e5fdbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:16.024924 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239a291-fc5d-490e-a8e3-acaddae71b46 service swift] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:16.025203 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239a291-fc5d-490e-a8e3-acaddae71b46 service swift] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:16.025474 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239a291-fc5d-490e-a8e3-acaddae71b46 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:16.033376 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239a291-fc5d-490e-a8e3-acaddae71b46 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:16.034894 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239a291-fc5d-490e-a8e3-acaddae71b46 service swift] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:16.043771 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1477/2933] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Sep 9 21:26:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 21:26:18.575879 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bbd196-e620-49e5-b3c8-24d050d2b69a None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:18.637495 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bbd196-e620-49e5-b3c8-24d050d2b69a None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:18.642021 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a54f0-8bcb-41e9-82c7-4d2af85da91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:18.642643 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a54f0-8bcb-41e9-82c7-4d2af85da91f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:18.642922 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a54f0-8bcb-41e9-82c7-4d2af85da91f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:18.643192 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a54f0-8bcb-41e9-82c7-4d2af85da91f admin admin] PATH_INFO: `/v3/users/e1d18999210f4a7d8ef45645e7781700` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:18.644344 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673a54f0-8bcb-41e9-82c7-4d2af85da91f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1d18999210f4a7d8ef45645e7781700)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:18.646818 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673a54f0-8bcb-41e9-82c7-4d2af85da91f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:18.789330 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:18.790791 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-673a54f0-8bcb-41e9-82c7-4d2af85da91f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1d18999210f4a7d8ef45645e7781700'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:18.791437 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-673a54f0-8bcb-41e9-82c7-4d2af85da91f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1d18999210f4a7d8ef45645e7781700'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:18.798356 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-673a54f0-8bcb-41e9-82c7-4d2af85da91f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1d18999210f4a7d8ef45645e7781700'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:18.810065 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-673a54f0-8bcb-41e9-82c7-4d2af85da91f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1d18999210f4a7d8ef45645e7781700'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:18.815650 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-673a54f0-8bcb-41e9-82c7-4d2af85da91f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1d18999210f4a7d8ef45645e7781700'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:18.925483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1457/2934] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:26:18 2022] DELETE /identity/v3/users/e1d18999210f4a7d8ef45645e7781700 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:18.930114 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50dc72-119d-4f30-a146-b8003e05ec68 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:18.930620 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50dc72-119d-4f30-a146-b8003e05ec68 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:18.930735 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50dc72-119d-4f30-a146-b8003e05ec68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:18.951204 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-cc50dc72-119d-4f30-a146-b8003e05ec68 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:18.952460 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cc50dc72-119d-4f30-a146-b8003e05ec68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:19.000457 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:19.012732 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1478/2935] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:19.022836 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a54f0-8bcb-41e9-82c7-4d2af85da91f admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:19.024281 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-673a54f0-8bcb-41e9-82c7-4d2af85da91f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:19.080594 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a54f0-8bcb-41e9-82c7-4d2af85da91f admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:19.084727 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf8078-c7e0-474a-b582-5d001cc03afd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:19.085326 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf8078-c7e0-474a-b582-5d001cc03afd service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:19.085600 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf8078-c7e0-474a-b582-5d001cc03afd service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:19.085870 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf8078-c7e0-474a-b582-5d001cc03afd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:19.094966 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf8078-c7e0-474a-b582-5d001cc03afd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:19.096416 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf8078-c7e0-474a-b582-5d001cc03afd service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:19.105694 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1458/2936] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:26:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:26:19.327816 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50dc72-119d-4f30-a146-b8003e05ec68 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:19.329295 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cc50dc72-119d-4f30-a146-b8003e05ec68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:19.342931 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50dc72-119d-4f30-a146-b8003e05ec68 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:19.348094 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2be96-be84-4857-8d4e-f019803e5dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:19.348858 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2be96-be84-4857-8d4e-f019803e5dca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:19.349149 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2be96-be84-4857-8d4e-f019803e5dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:19.349422 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2be96-be84-4857-8d4e-f019803e5dca admin admin] PATH_INFO: `/v3/projects/ccdaeb5238614decb2020f32a52c0b20` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:19.350602 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e2be96-be84-4857-8d4e-f019803e5dca admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccdaeb5238614decb2020f32a52c0b20)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:19.352613 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e2be96-be84-4857-8d4e-f019803e5dca admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:19.457828 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f4e2be96-be84-4857-8d4e-f019803e5dca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccdaeb5238614decb2020f32a52c0b20', 'request_id': 'req-f4e2be96-be84-4857-8d4e-f019803e5dca'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:19.464354 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f4e2be96-be84-4857-8d4e-f019803e5dca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccdaeb5238614decb2020f32a52c0b20', 'request_id': 'req-f4e2be96-be84-4857-8d4e-f019803e5dca'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:19.544402 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f4e2be96-be84-4857-8d4e-f019803e5dca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccdaeb5238614decb2020f32a52c0b20', 'request_id': 'req-f4e2be96-be84-4857-8d4e-f019803e5dca'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:19.555575 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f4e2be96-be84-4857-8d4e-f019803e5dca admin admin] The token cache is being invalidate because project ccdaeb5238614decb2020f32a52c0b20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:26:19.555864 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f4e2be96-be84-4857-8d4e-f019803e5dca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:19.556504 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f4e2be96-be84-4857-8d4e-f019803e5dca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:19.557960 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1479/2937] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:26:19 2022] DELETE /identity/v3/projects/ccdaeb5238614decb2020f32a52c0b20 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:19.562787 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9913b2-0c80-4e33-98c7-0b8e48bfcded None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:19.563070 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9913b2-0c80-4e33-98c7-0b8e48bfcded None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:19.563342 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9913b2-0c80-4e33-98c7-0b8e48bfcded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:19.584491 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-dc9913b2-0c80-4e33-98c7-0b8e48bfcded None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:19.585687 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-dc9913b2-0c80-4e33-98c7-0b8e48bfcded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:19.644536 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1459/2938] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:26:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:19.652216 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde319a6-ab99-4c7c-99ef-069de6867e64 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:19.652504 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde319a6-ab99-4c7c-99ef-069de6867e64 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:19.652779 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde319a6-ab99-4c7c-99ef-069de6867e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:19.673229 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-cde319a6-ab99-4c7c-99ef-069de6867e64 None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:19.674423 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cde319a6-ab99-4c7c-99ef-069de6867e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:19.683261 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1480/2939] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:26:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:19.687328 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9913b2-0c80-4e33-98c7-0b8e48bfcded None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:19.697207 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9913b2-0c80-4e33-98c7-0b8e48bfcded None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:19.701851 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae593e-2d31-471f-98e1-4e422766041d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:19.702442 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae593e-2d31-471f-98e1-4e422766041d admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:19.702725 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae593e-2d31-471f-98e1-4e422766041d admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:19.703000 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae593e-2d31-471f-98e1-4e422766041d admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:19.703646 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae593e-2d31-471f-98e1-4e422766041d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:26:19.703952 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae593e-2d31-471f-98e1-4e422766041d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:19.705580 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae593e-2d31-471f-98e1-4e422766041d admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:19.717222 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1460/2940] 10.222.0.48 () {58 vars in 1153 bytes} [Fri Sep 9 21:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:26:19.721448 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde319a6-ab99-4c7c-99ef-069de6867e64 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:19.730985 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde319a6-ab99-4c7c-99ef-069de6867e64 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:19.735726 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c4a49-e1cd-4651-8d77-26b2b3f18c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:19.736340 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c4a49-e1cd-4651-8d77-26b2b3f18c98 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:19.736619 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c4a49-e1cd-4651-8d77-26b2b3f18c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:19.736886 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c4a49-e1cd-4651-8d77-26b2b3f18c98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:19.737500 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1c4a49-e1cd-4651-8d77-26b2b3f18c98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:19.739499 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1c4a49-e1cd-4651-8d77-26b2b3f18c98 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:19.818142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1481/2941] 10.222.0.48 () {60 vars in 1153 bytes} [Fri Sep 9 21:26:19 2022] POST /identity/v3/projects => generated 399 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:26:19.822483 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae593e-2d31-471f-98e1-4e422766041d admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:19.832604 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae593e-2d31-471f-98e1-4e422766041d admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:19.880753 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:19.889663 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad6eda8-4881-4281-8980-6219dd829ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:19.893556 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6eda8-4881-4281-8980-6219dd829ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:19.893556 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6eda8-4881-4281-8980-6219dd829ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:19.893556 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6eda8-4881-4281-8980-6219dd829ce6 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:19.893556 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad6eda8-4881-4281-8980-6219dd829ce6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:19.893969 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad6eda8-4881-4281-8980-6219dd829ce6 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:19.973301 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1461/2942] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Sep 9 21:26:19 2022] POST /identity/v3/users => generated 300 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:26:19.977955 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c4a49-e1cd-4651-8d77-26b2b3f18c98 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:19.988755 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c4a49-e1cd-4651-8d77-26b2b3f18c98 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:19.993534 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ccca1e-8d3c-4225-9220-17e7b969c77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:19.996780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ccca1e-8d3c-4225-9220-17e7b969c77e admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:19.996780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ccca1e-8d3c-4225-9220-17e7b969c77e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:19.996780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ccca1e-8d3c-4225-9220-17e7b969c77e admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:19.996780 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ccca1e-8d3c-4225-9220-17e7b969c77e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:19.997194 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ccca1e-8d3c-4225-9220-17e7b969c77e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:20.007152 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1482/2943] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:20.011187 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad6eda8-4881-4281-8980-6219dd829ce6 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:20.022183 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad6eda8-4881-4281-8980-6219dd829ce6 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:20.026906 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfaf4ec-eb2d-4834-ba71-6a9fb749555d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:20.027503 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfaf4ec-eb2d-4834-ba71-6a9fb749555d admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:20.027782 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfaf4ec-eb2d-4834-ba71-6a9fb749555d admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:20.028055 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfaf4ec-eb2d-4834-ba71-6a9fb749555d admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:20.028736 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfaf4ec-eb2d-4834-ba71-6a9fb749555d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:20.030642 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfaf4ec-eb2d-4834-ba71-6a9fb749555d admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:20.042167 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1462/2944] 10.222.0.48 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:20.046315 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ccca1e-8d3c-4225-9220-17e7b969c77e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:20.056309 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ccca1e-8d3c-4225-9220-17e7b969c77e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:20.060846 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51a599-9861-49b3-825b-b9f1a92f11ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:20.061477 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51a599-9861-49b3-825b-b9f1a92f11ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:20.061754 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51a599-9861-49b3-825b-b9f1a92f11ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:20.062021 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51a599-9861-49b3-825b-b9f1a92f11ba admin admin] PATH_INFO: `/v3/projects/006a87279868415e9b2b193765d4551a/users/b6702e0d406a426589360f5e83f7b7e5/roles/c59d891d042e48bab625f7b3666ef536` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:20.077350 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc51a599-9861-49b3-825b-b9f1a92f11ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=006a87279868415e9b2b193765d4551a, user_id=b6702e0d406a426589360f5e83f7b7e5, role_id=c59d891d042e48bab625f7b3666ef536)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:20.079967 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc51a599-9861-49b3-825b-b9f1a92f11ba admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:20.162923 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1483/2945] 10.222.0.48 () {60 vars in 1483 bytes} [Fri Sep 9 21:26:20 2022] PUT /identity/v3/projects/006a87279868415e9b2b193765d4551a/users/b6702e0d406a426589360f5e83f7b7e5/roles/c59d891d042e48bab625f7b3666ef536 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:20.172921 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6facc75d-e5bc-4155-bd60-0afcbe00386d None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:20.173265 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6facc75d-e5bc-4155-bd60-0afcbe00386d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:20.173484 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6facc75d-e5bc-4155-bd60-0afcbe00386d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:20.200392 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-6facc75d-e5bc-4155-bd60-0afcbe00386d None None] MFA Rules not processed for user `b6702e0d406a426589360f5e83f7b7e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:20.201592 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-6facc75d-e5bc-4155-bd60-0afcbe00386d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:20.266560 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1463/2946] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:26:20 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:20.278414 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51a599-9861-49b3-825b-b9f1a92f11ba admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:20.280096 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fc51a599-9861-49b3-825b-b9f1a92f11ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:20.335814 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51a599-9861-49b3-825b-b9f1a92f11ba admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:20.339660 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40655bba-3216-4906-8d9f-498425366ac3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf91131ee5bb481bbb47097cad0c003a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:20.340276 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40655bba-3216-4906-8d9f-498425366ac3 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:20.340556 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40655bba-3216-4906-8d9f-498425366ac3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:20.340826 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40655bba-3216-4906-8d9f-498425366ac3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:20.349210 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40655bba-3216-4906-8d9f-498425366ac3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:20.350622 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40655bba-3216-4906-8d9f-498425366ac3 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:20.359439 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1484/2947] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:26:20 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:26:29.341843 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6facc75d-e5bc-4155-bd60-0afcbe00386d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:29.404053 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6facc75d-e5bc-4155-bd60-0afcbe00386d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:29.408877 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eafb9c8-1e4e-4f0c-b12e-9a5859840f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:29.413961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eafb9c8-1e4e-4f0c-b12e-9a5859840f0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:29.413961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eafb9c8-1e4e-4f0c-b12e-9a5859840f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:29.413961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eafb9c8-1e4e-4f0c-b12e-9a5859840f0c admin admin] PATH_INFO: `/v3/users/b6702e0d406a426589360f5e83f7b7e5` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:29.413961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eafb9c8-1e4e-4f0c-b12e-9a5859840f0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6702e0d406a426589360f5e83f7b7e5)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:29.413961 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eafb9c8-1e4e-4f0c-b12e-9a5859840f0c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:29.569779 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8eafb9c8-1e4e-4f0c-b12e-9a5859840f0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6702e0d406a426589360f5e83f7b7e5'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:29.570359 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8eafb9c8-1e4e-4f0c-b12e-9a5859840f0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6702e0d406a426589360f5e83f7b7e5'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:29.577389 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8eafb9c8-1e4e-4f0c-b12e-9a5859840f0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6702e0d406a426589360f5e83f7b7e5'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:29.582596 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:29.589841 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8eafb9c8-1e4e-4f0c-b12e-9a5859840f0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6702e0d406a426589360f5e83f7b7e5'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:29.591682 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8eafb9c8-1e4e-4f0c-b12e-9a5859840f0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6702e0d406a426589360f5e83f7b7e5'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:29.665891 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1464/2948] 10.222.0.48 () {58 vars in 1224 bytes} [Fri Sep 9 21:26:29 2022] DELETE /identity/v3/users/b6702e0d406a426589360f5e83f7b7e5 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:29.670486 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7a4fd-6cfc-4ec6-97b3-9902d4ef1bad None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:29.670982 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7a4fd-6cfc-4ec6-97b3-9902d4ef1bad None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:29.671086 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7a4fd-6cfc-4ec6-97b3-9902d4ef1bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:29.690599 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-cdb7a4fd-6cfc-4ec6-97b3-9902d4ef1bad None None] MFA Rules not processed for user `2e3a7a2557844cc78377f788a7c8c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:29.691791 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cdb7a4fd-6cfc-4ec6-97b3-9902d4ef1bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:29.704028 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:29.749454 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1485/2949] 10.222.0.48 () {58 vars in 958 bytes} [Fri Sep 9 21:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:29.759756 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eafb9c8-1e4e-4f0c-b12e-9a5859840f0c admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:29.761210 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8eafb9c8-1e4e-4f0c-b12e-9a5859840f0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:29.817706 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eafb9c8-1e4e-4f0c-b12e-9a5859840f0c admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:29.822094 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63bbe26-e5eb-4c6e-b7e6-9083da164f3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5dfac06ac9492a86503ba133c638a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:29.822636 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63bbe26-e5eb-4c6e-b7e6-9083da164f3e service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:29.822911 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63bbe26-e5eb-4c6e-b7e6-9083da164f3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:29.823181 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63bbe26-e5eb-4c6e-b7e6-9083da164f3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:29.832025 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63bbe26-e5eb-4c6e-b7e6-9083da164f3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:29.833478 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63bbe26-e5eb-4c6e-b7e6-9083da164f3e service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:29.843113 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1465/2950] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:26:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:26:30.374673 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7a4fd-6cfc-4ec6-97b3-9902d4ef1bad None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:30.376142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cdb7a4fd-6cfc-4ec6-97b3-9902d4ef1bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:30.390836 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7a4fd-6cfc-4ec6-97b3-9902d4ef1bad None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:30.396022 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a66198-f93c-4417-870a-853579c88848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3a7a2557844cc78377f788a7c8c3ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd43f38801344407291af57c99bc81066', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:30.396790 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a66198-f93c-4417-870a-853579c88848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:30.397073 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a66198-f93c-4417-870a-853579c88848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:30.397856 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a66198-f93c-4417-870a-853579c88848 admin admin] PATH_INFO: `/v3/projects/006a87279868415e9b2b193765d4551a` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:30.399147 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a66198-f93c-4417-870a-853579c88848 admin admin] RBAC: Authorizing `identity:delete_project(project_id=006a87279868415e9b2b193765d4551a)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:30.401142 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a66198-f93c-4417-870a-853579c88848 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:30.496578 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a8a66198-f93c-4417-870a-853579c88848 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '006a87279868415e9b2b193765d4551a', 'request_id': 'req-a8a66198-f93c-4417-870a-853579c88848'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:30.502808 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a8a66198-f93c-4417-870a-853579c88848 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '006a87279868415e9b2b193765d4551a', 'request_id': 'req-a8a66198-f93c-4417-870a-853579c88848'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:30.571099 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a8a66198-f93c-4417-870a-853579c88848 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '006a87279868415e9b2b193765d4551a', 'request_id': 'req-a8a66198-f93c-4417-870a-853579c88848'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:30.581789 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a8a66198-f93c-4417-870a-853579c88848 admin admin] The token cache is being invalidate because project 006a87279868415e9b2b193765d4551a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:26:30.582093 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a8a66198-f93c-4417-870a-853579c88848 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:30.582716 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a8a66198-f93c-4417-870a-853579c88848 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:30.584226 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1486/2951] 10.222.0.48 () {58 vars in 1233 bytes} [Fri Sep 9 21:26:30 2022] DELETE /identity/v3/projects/006a87279868415e9b2b193765d4551a => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:39.838925 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=32,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:40.578723 dv-cinder-831247-28-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:29:44.104364 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63bbe26-e5eb-4c6e-b7e6-9083da164f3e service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:29:44.106075 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e63bbe26-e5eb-4c6e-b7e6-9083da164f3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:29:44.174492 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63bbe26-e5eb-4c6e-b7e6-9083da164f3e service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:29:44.241326 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6329fb-ba3d-409a-8108-e9cf5eb2ce38 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4f6de2065d4fbea3fa141da760f708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91319aa782334fb9930770029b74e84a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:29:44.241945 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6329fb-ba3d-409a-8108-e9cf5eb2ce38 service placement] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:44.242229 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6329fb-ba3d-409a-8108-e9cf5eb2ce38 service placement] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:44.242506 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6329fb-ba3d-409a-8108-e9cf5eb2ce38 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:44.244829 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-cc6329fb-ba3d-409a-8108-e9cf5eb2ce38 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:29:44.255975 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6329fb-ba3d-409a-8108-e9cf5eb2ce38 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:29:44.257575 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6329fb-ba3d-409a-8108-e9cf5eb2ce38 service placement] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:29:44.267572 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1466/2952] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Sep 9 21:29:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:29:54.263203 dv-cinder-831247-28-iscsi devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=58 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}